PaperlayerView exact revision on arXiv

Source-linked AI summary

The Independence Prior of SAEs Fragments Visual Concepts

Tommaso Mencattini, Giorgos Nikolaou, Donato Crisostomi, Thomas Fel, Francesco Montagna, Emanuele Rodolà, Francesco Locatello

arXiv:2610.04112v1cs.AI

TL;DR

Tokenwise SAEs treat patch codes as independent, despite spatial dependencies in vision-model activations. The paper introduces MF-LRH and derives SPATIAL-SAE as amortized MAP inference under it. The method improves synthetic concept recovery and real-activation interpretability, with a reconstruction tradeoff.

  • Problem

    Tokenwise SAE objectives impose patch independence, although visual concepts and vision-model activations exhibit spatial dependencies.

    Source: §1 · p. 1, §3.1 · p. 4

  • Method

    MF-LRH splits patch codes into neighbor-coupled smooth fields and sparse local innovations, and SPATIAL-SAE amortizes MAP inference under this model.

    Source: §1 · p. 2, §4.1 · p. 6

  • Results

    SPATIAL-SAE improves synthetic concept recovery in 288 out of 300 settings and improves every evaluated interpretability proxy on real vision-model activations.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    The spatial prior moves explanations toward more interpretable features and coherent activation maps while trading off reconstruction.

    Source: §6 · p. 9

Abstract

from arXiv · show

Sparse Autoencoders (SAEs) decompose model activations into sparse combinations of interpretable dictionary atoms. Although SAEs are grounded in the Linear Representation Hypothesis (LRH), their objective smuggles in an additional prior: concepts across patches are treated as independent, an assumption clearly violated by natural images and by the activations they induce. We therefore specialize LRH to vision through the Markov-Field Linear Representation Hypothesis (MFLRH), which adds the missing spatial dependencies to the LRH assumptions. We thus propose Spatial-SAE as an amortized MAP estimator under the MFLRH. Spatial-SAE consistently outperforms standard SAEs in concept recovery and interpretability. Across four variants, it achieves a 96% average win rate on synthetic concept recovery and improves interpretability on DINOv2 activations, at a reconstruction cost concentrated in high spatial frequencies.

1 INTRODUCTION

Vision-model SAEs often fragment visual concepts because their patch-independent objective cannot reward spatially coherent explanations. The paper replaces that prior with MF-LRH and derives SPATIAL-SAE, which improves synthetic concept recovery and real-activation interpretability while incurring a frequency-concentrated reconstruction cost.

Motivation

  • Visual SAE features can split one concept across atoms, leave holes within objects, and assign neighboring patches to different atoms, making them harder to interpret and trust.The introduction cites Chanin et al. (2025b), Ayonrinde et al. (2024), Chanin et al. (2025a), and Bhalla et al. (2026) on fragmentation.Source: §1 · p. 1
  • Tokenwise SAEs implicitly treat neighboring patch concepts as independent, although objects, textures, and regions extend across adjacent patches.The introduction cites Lubana et al. (2026) for the SAE MAP interpretation and Elhage et al. (2022), Park et al. (2024), and Fel et al. (2026) for LRH.Source: §1 · p. 1
  • The paper identifies the patch-independence prior as one of two fragmentation pressures, alongside the rate–distortion–polysemanticity tradeoff.At fixed sparsity, reconstruction can favor polysemantic atoms that capture pieces of concepts rather than whole concepts.Source: §1 · p. 1

Contributions

  • MF-LRH adds neighboring-patch dependencies to LRH by splitting each code into a neighbor-shared smooth component and sparse local innovations.Innovations capture local evidence such as boundaries and small objects.Source: §1 · p. 2
  • SPATIAL-SAE improves synthetic concept recovery in 288 out of 300 settings and produces more concept-aligned, region-shaped features on frozen DINOv2 activations.The reconstruction cost is concentrated in high spatial frequencies.Source: §1 · p. 2

2 BACKGROUND

The background frames concept extraction as recovering a sparse dictionary representation of model activations. Sparse coding solves this inverse problem directly, while an SAE amortizes code inference with a learned encoder.

Linear Representation Hypothesis

  • LRH models an activation as an approximately sparse linear combination of directions in an overcomplete, approximately non-redundant dictionary.The formal definition additionally specifies overcompleteness and quasi-orthogonality conditions.Source: §2 · p. 3
  • Applied across activations, LRH gives H ≈ ZD, making concept extraction an inverse problem of recovering dictionary directions D and sparse activation codes Z.The passage cites Hindupur et al. (2025) for this inverse-problem framing.Source: §2 · p. 3

Sparse coding and SAEs

  • Sparse coding jointly optimizes a dictionary and sparse codes, whereas an SAE uses a shared encoder to predict codes in one forward pass.The encoder can return a suboptimal code, with approximation cost measured by the amortization gap.Source: §2 · p. 3

3 SPATIAL STRUCTURE IN VISION-MODEL ACTIVATIONS

The section argues that spatially dependent vision activations are poorly matched to tokenwise SAE objectives, which do not account for relations among patches. MF-LRH instead couples complementary smooth and innovation fields across a patch graph.

3.1 SAES ADD A PATCH-INDEPENDENCE PRIOR

  • A tokenwise SAE objective is invariant to jointly permuting activations and codes, so it assigns no extra cost when a coherent patch layout is arbitrarily rearranged.Consequently, it has no criterion for preferring spatially consistent explanations.Source: §3.1 · p. 4
  • Vision-transformer patch activations retain spatial dependence: true neighbors explain more variance than shuffled same-image neighborhoods, and nearby patches remain more similar across network depth.PCA visualizations also show coherent spatial patterns; the measured dependence motivates a prior that uses neighborhood information.Source: §3.1 · p. 4, Figure 2 · p. 4

3.2 MARKOV-FIELD LINEAR REPRESENTATION HYPOTHESIS

  • MF-LRH decomposes each patch code as Z = S + R, with attractive spatial coupling for smooth shared content and repulsive coupling for local innovations.The smooth field represents content shared across patches; the innovation field captures local departures.Source: §3.2 · p. 5
  • MF-LRH retains LRH while replacing patch independence with complementary local dependencies; at μ = 0, it factorizes across patches and ordinary sparse coding is recovered when S ≡ 0 and R = Z.Source: §3.2 · p. 5

4 SPATIAL SPARSE AUTOENCODERS

The method derives SPATIAL-SAE as amortized MAP inference under MF-LRH, then uses an encoder to infer shared smooth codes from neighboring patches and local innovations from the residual.

4.1 SPATIAL-SAE IS AMORTIZED MAP INFERENCE UNDER MF-LRH

  • Under a Gaussian observation model and fixed dictionary and graph, MAP inference under MF-LRH becomes graph-regularized sparse coding with reconstruction, sparsity, and spatial-interaction terms.The smooth-field term suppresses high graph frequencies, while the innovation term suppresses low frequencies and promotes spatial contrast.Source: §4.1 · p. 6
  • SPATIAL-SAE replaces per-image optimization with a learned encoder and adds the spatial prior through the training loss, making it a plug-and-play extension of SAE families.The graph terms make a patch's explanation cost depend on how its neighbors are explained.Source: §4.1 · p. 6

4.2 ESTIMATING THE SMOOTH AND INNOVATION FIELDS

  • The encoder estimates each smooth code by attending to neighboring activations, then predicts the innovation code from the residual after smooth reconstruction.The smooth-code estimate does not see the target patch; stop-gradient prevents the innovation stream from backpropagating through the smooth reconstruction.Source: §4.2 · p. 7

5 EXPERIMENTS

The experiments test SPATIAL-SAE on synthetic data with known concepts and on frozen DINOv2 activations. Across both settings, the paper evaluates interpretability alongside reconstruction, which can trade off against it.

5.1 CONCEPT RECOVERY ON SYNTHETIC DATA

  • The synthetic benchmark uses known dictionaries and codes to measure concept recovery across a grid sweep of smoothness and innovation parameters, with 300 matched runs per architecture.The reported measures are MCC, support F1, held-out R², and polysemanticity.Source: §5.1 · pp. 7–8
  • Across TopK and BatchTopK, SPATIAL-SAE wins every MCC, F1, and polysemanticity comparison; JumpReLU wins every F1 and R² comparison.Against the otherwise identical unregularized two-stream model, tuned spatial regularization improves F1 by up to 6.9 percentage points.Source: Table 1 · p. 8, §5.1 · p. 8, Figure 5 · p. 8

5.2 CONCEPT DISCOVERY IN VISION-MODEL ACTIVATIONS

  • On frozen DINOv2 ViT-B/14 activations from ADE20K, four sparsifiers are compared using semantic and spatial alignment measures and blinded autointerpretability ratings.The evaluation includes Normalized F1, Region Coherence, Map Quality, and Monosemanticity.Source: §5.2 · pp. 8–9
  • Across sparsifiers, SPATIAL-SAE improves every interpretability measure; for TopK, Normalized F1 increases from 57.5 to 65.4 and Region Coherence from 15.5 to 25.9.The gains trade off against reconstruction, with the R² cost concentrated at high spatial frequencies; varying regularization traces a Normalized-F1–R² frontier.Source: Figure 6 · p. 9, §5.2 · p. 9

6 CONCLUSION

The section frames SAEs as amortized approximations to sparse coding motivated by the LRH, then describes Spatial-SAE’s shift from independent patch codes to spatially coupled codes and its interpretability–reconstruction tradeoff.

6 CONCLUSION

  • SPATIAL-SAE improves synthetic concept recovery and reduces polysemanticity, improves every evaluated real-activation proxy, and moves along the RDP frontier toward interpretability at a reconstruction cost.On real vision-model activations, concept recovery and polysemanticity lack a ground-truth dictionary, so the paper evaluates proxies.Source: §6 · p. 9

A.1 PATCH GRAPHS, LAPLACIAN, AND DIRICHLET ENERGY

  • Graph Dirichlet energy measures variation across neighboring patch codes: minimizing it encourages agreement, while stabilized negative energy favors contrast without unbounded magnitude growth.The shifted quadratic combines a negative edge term with a magnitude term to stabilize contrast-favoring interactions.Source: §A.1 · pp. 15–16

A.2 UNDIRECTED GRAPHICAL MODELS AND MARKOV RANDOM FIELDS

  • Undirected graphical models encode local patch dependencies through pairwise potentials, letting attraction favor neighboring agreement and repulsion favor contrast.A patch code may interact directly with its neighbors without requiring direct interactions with every other patch.Source: §A.2 · pp. 17–18

B.1 FROM LRH TO SPARSE CODING AND SPARSE AUTOENCODERS

  • LRH frames concept extraction as sparse dictionary learning, while an SAE amortizes per-activation sparse inference by learning a shared encoder to predict codes.The shared encoder replaces a separate sparse-inference optimization for each activation.Source: §B.1 · pp. 20–21
  • An SAE amortizes LRH-induced sparse inference by learning an encoder that returns codes instead of solving separately for each activation.Its outer objective retains the reconstruction–sparsity tradeoff of sparse dictionary learning, while the encoder constrains the returned codes.Source: §B.1 · pp. 21–22
  • The encoder makes code generation fast, but its restricted range can prevent it from returning the exact or better sparse codes available through more expressive inference.This approximation cost is the amortization gap; it can remain nonzero for standard SAEs even when better sparse codes exist.Source: §B.1 · pp. 21–22
  • The LRH motivates sparse, overcomplete dictionary learning; sparse coding aligns with that hypothesis, and an SAE provides an amortized approximation to sparse coding.Source: §B.1 · p. 22

B.2 THE MAP INTERPRETATION OF VANILLA SAES

The Bayesian view shows that sparse coding is the MAP estimator for noisy LRH, while the vanilla tokenwise objective adds an independence prior across patch codes. MF-LRH replaces that spatially factorized prior with local interactions that favor persistent smooth content and localized innovations.

B.2 THE MAP INTERPRETATION OF VANILLA SAES

  • Under noisy LRH, sparse coding is the MAP estimator; vanilla SAEs approximate this estimator through amortized inference.The likelihood supplies reconstruction error and the sparsity-promoting prior supplies the code penalty.Source: §B.2 · pp. 22–23
  • The vanilla tokenwise sparsity objective induces a factorized prior over patch codes, so it does not encode which patches are neighbors.The same limitation applies across common tokenwise variants, including ReLU, TopK, BatchTopK, and JumpReLU.Source: §B.2 · pp. 22, 25
  • Row-additive reconstruction and sparsity terms cannot distinguish a coherent patch layout from one with activations and codes jointly permuted.This follows because the objective has no graph term that preserves neighborhood relations.Source: §B.2 · p. 25

B.3 THE MF-LRH PRIOR

  • MF-LRH retains the linear representation while splitting codes into an attractive smooth field for shared content and a repulsive innovation field for local departures.The spatial penalties favor persistent structure in S and local variation in R; reconstruction supplies evidence for needed innovations.Source: §B.3 · pp. 25–26
  • The continuous MF-LRH prior is normalizable on the real code space or the nonnegative orthant.Its smooth and innovation fields have attractive and repulsive pairwise interactions, respectively.Source: §B.3 · p. 26

B.4 FROM MF-LRH TO SPATIAL-SAE

The paper derives SPATIAL-SAE by taking the MAP estimator under the MF-LRH prior and amortizing it with an encoder. Its graph dependence uses spatial relations rather than arbitrary patch indices.

B.4 FROM MF-LRH TO SPATIAL-SAE

  • Under the noisy observation model, the MAP objective combines a reconstruction term with the MF-LRH prior energy.The Gaussian likelihood supplies reconstruction error, while the prior supplies the spatial and sparsity penalties.Source: §B.4 · p. 27
  • SPATIAL-SAE amortizes the spatial estimator by using a shared encoder to predict smooth and innovation fields from each image.The encoder predictions approximate the spatial MAP estimator and can incur an amortization gap.Source: §B.4 · p. 27
  • Simultaneously relabeling activations, fields, and the graph preserves the objective, so its graph dependence is not tied to patch indexing.With the graph fixed and μ > 0, objective-preserving permutations must preserve its weighted adjacency.Source: §B.4 · pp. 27–28

C SYNTHETIC EXPERIMENTS

Because real vision-model activations do not reveal their generating dictionary or latent codes, the experiments use a controlled synthetic benchmark with known concepts. The section reports component, geometry, and capacity results.

C SYNTHETIC EXPERIMENTS

  • Real vision-model activations cannot directly establish whether an SAE recovers the underlying concepts because their generating dictionary and latent codes are unknown.The experiments therefore use a controlled benchmark with known concept directions and firing patterns.Source: C SYNTHETIC EXPERIMENTS · p. 29
  • The synthetic benchmark is used to evaluate concept recovery under controlled ground truth.The passage identifies the known directions and firing patterns as the benchmark’s controlled elements.Source: C SYNTHETIC EXPERIMENTS · p. 29
  • The section reports component, geometry, and capacity results for the synthetic benchmark.The benchmark is described in Section C.1 and its results in Section C.2.Source: C SYNTHETIC EXPERIMENTS · p. 29

C.1 EXPERIMENTAL SETUP

The synthetic setup varies smooth and innovation structure and compares matched-width, matched-sparsity models across geometry and capacity grids. Its sweeps assess performance using concept-recovery metrics and polysemanticity.

C.1 EXPERIMENTAL SETUP

  • The synthetic activations combine smooth concepts with sparse local innovations, testing the spatial prior without sampling from its exact assumed distribution.This tests whether the prior remains useful when data share the relevant structure but not the assumed distribution.Source: §C.1 · p. 29
  • Smooth-stream coupling τ controls spatial coherence, while support quantile q controls active area; innovation radius r controls packet extent.The streams are sampled independently, so their active concepts and spatial supports may overlap.Source: §C.1 · pp. 30–31
  • The matched-width, matched-sparsity comparison uses B for a standard single-stream SAE, D for decomposition without trace penalties, and DT for D with trace penalties.B → D isolates access to decomposition; D → DT isolates the trace prior’s incremental effect.Source: §C.1 · pp. 31–32
  • The geometry grid varies τ, q, and r, while the 100-cell capacity grid varies L0, activation dimension d, and dictionary narrowness ν.The geometry and capacity grids together contain 159 distinct configurations.Source: §C.1 · p. 32
  • The sweeps compare MCC, support F1, held-out R2, and polysemanticity, counting strict metric improvements as wins.The combined geometry and capacity results are summarized as strict win rates in Table 2.Source: Table 2 · p. 31, §C.1 · p. 33

C.2 EFFECTS OF DECOMPOSITION AND SPECTRAL REGULARIZATION

The component ablations show that decomposition and spectral regularization improve recovery metrics, but the trace prior’s benefit depends on geometry, capacity, and its strength.

Component ablation

  • D beats B in 99%/99%/98% of configurations on MCC/polysemanticity/F1; at low trace strength, DT beats D in 96%/96%/91%, with the full ladder in 96%/94%/89%.The ladder weakens as trace strength increases, while R2 rarely follows it; the trace coefficient must be tuned for recovery rather than reconstruction.Source: §C.2 · p. 33

Geometry Sweep

  • Across the geometry grid, DT improves F1 over D throughout and improves polysemanticity throughout at low and medium trace strengths, with no high-strength deterioration.Against B, DT improves polysemanticity and F1 throughout the grid at every trace strength.Source: §C.2 · p. 33

Capacity Sweep

  • At high trace strength, DT worsens polysemanticity in 49% and F1 in 33% of configurations against D, making mean polysemanticity change slightly unfavorable.At low strength, DT improves both plotted metrics over B in 100% of configurations; sensitivity is stronger against D.Source: §C.2 · p. 34

D EXPERIMENTS WITH NATURAL IMAGES

The natural-image experiments evaluate learned concepts with dense scene annotations on ADE20K and COCO-STUFF. The appendix documents the datasets, frozen vision backbone, implementation, and sparsity-budget results.

D EXPERIMENTS WITH NATURAL IMAGES

  • The appendix gives the full natural-image experimental account, including datasets, backbone, implementation, evaluation metrics, interpretability protocol, ablations, transfer, and qualitative results.Training-epoch times vary by model and dataset, from about four minutes for static and convolutional spatial models on ADE20K to 28–35 minutes for COCO-STUFF context attention.Source: D EXPERIMENTS WITH NATURAL IMAGES · pp. 35–36
  • ADE20K and COCO-STUFF provide dense pixel labels across 150 and 171 semantic classes, respectively, enabling patch-level concept evaluation and testing transfer across datasets.Experiments use frozen DINOv2 ViT-B/14 patch activations on a 16 × 16 grid; configurations selected on ADE20K are tested on COCO-STUFF without retuning.Source: D EXPERIMENTS WITH NATURAL IMAGES · p. 35
  • At budget 16, SPATIAL-SAE has higher Normalized F1 throughout; it improves baseline reconstruction over budget 8 while retaining at least 98% of each baseline’s best Normalized F1, and is optimal for BatchTopK.Table 3 reports means and standard deviations over five seeds; Normalized F1 is relative to the corresponding linear probe.Source: Table 3 · p. 36

D.1 IMPLEMENTATION DETAILS

SPATIAL-SAE uses separate smooth and innovation streams, with neighbor-based smooth-field estimation and spectral regularization. The implementation matches total dictionary width and sparsity budgets to baseline models.

D.1 IMPLEMENTATION DETAILS

  • Training runs for ten epochs with Adam; spatial models process complete patch grids, while baselines receive shuffled patch batches, and final checkpoints are used without early stopping.Sweep configurations use seed 0; selected models are retrained with seeds 0–4.Source: §D.1 · p. 36
  • Models use linear encoders with ReLU followed by the selected sparsifier, and dictionary atoms are renormalized to unit ℓ2 norm on each forward pass.Unless otherwise stated, each model uses 3,072 dictionary atoms.Source: §D.1 · p. 36
  • The smooth and innovation streams share the baseline’s total dictionary width and sparsity budget, allocated in proportion to their budgets; a 14:2 split assigns 2,688 and 384 atoms.Seven stream-budget splits are swept, from 2:14 through 14:2.Source: §D.1 · p. 36
  • The target is 16 active features per patch; controllers adjust ReLU and JumpReLU sparsity penalties, and reanimation regularization encourages inactive atoms to fire.For final runs targeting E[L0] = 16, active-feature counts differ from target by at most one, and fewer than 0.4% of atoms are inactive on the evaluation pool.Source: §D.1 · p. 37
  • Context attention estimates each patch’s smooth code from neighboring activations without observing the target patch, while the innovation encoder processes the residual with gradients stopped through the smooth reconstruction.Static and convolutional alternatives test target-patch projection and neighbor aggregation, respectively; both retain the two-stream structure.Source: §D.1 · pp. 37–38
  • The spectral loss penalizes high-frequency energy in the smooth stream and low-frequency energy in the innovation stream; setting µ to zero removes both penalties.The Laplacian eigendecomposition is computed once and reused for regularization and spectral metrics.Source: §D.1 · p. 38

D.2 EVALUATION METRICS AND SPECTRAL DIAGNOSTICS

The evaluation combines reconstruction, feature-to-concept alignment, spatial coherence, and graph-frequency diagnostics. A separate blinded interpretability protocol scores whether selected features represent coherent concepts and localize them well.

D.2 EVALUATION METRICS AND SPECTRAL DIAGNOSTICS

  • Held-out metrics are aggregated over evaluation samples and then averaged across training seeds; SPATIAL-SAE feature measures use the concatenated smooth and innovation codes.The evaluation treats reconstruction, feature alignment, and complete-code spatial coherence as complementary measures because real concepts are unobserved.Source: §D.2 · p. 38
  • Concept F1 compares thresholded feature activation masks with annotated class masks; Normalized F1 expresses the best feature’s score relative to a linear probe for reliably decoded classes.Normalized F1 measures how much of a class’s linearly decodable information is captured by one SAE feature.Source: §D.2 · p. 39
  • Region coherence compares code similarity within and across annotated regions at matched patch distances, so a larger score indicates more faithful preservation of semantic regions.The matched-distance comparison controls for same-region patches generally being closer together.Source: §D.2 · pp. 39–40
  • Concept-relative spectra compare normalized graph-frequency energy in learned codes with energy in concept masks; positive values indicate excess code energy at a frequency and negative values indicate under-representation.The comparison tests whether learned codes reflect the spatial scales present in annotated concepts.Source: §D.2 · pp. 40–41
  • Per-feature graph frequency is an activation-energy-weighted mean: low values indicate smooth maps, while high values indicate rapidly varying maps.The diagnostic complements whole-code spectra by showing frequency distribution across individual features and decoder directions.Source: §D.2 · p. 41

D.3 AUTOMATED INTERPRETABILITY

  • For automated interpretability, a blinded rater scores seven top-activating images per class for concept consistency and map quality, using panels with overlaid activation maps.Monosemanticity scores whether one concept explains highlighted regions across images; map quality scores their coverage, tightness, and consistency.Source: §D.2 · p. 39, §D.3 · pp. 41–42
  • ADE20K contributes 132 classes and COCO-STUFF 171; 95% class-bootstrap intervals use 2,000 resamples, and two COCO panels received duplicate ratings with identical scores that were averaged.The intervals quantify variation across semantic classes, not across seeds, ratings, or raters.Source: §D.3 · p. 42

D.4 ABLATIONS, MODEL SELECTION, AND ADDITIONAL RESULTS

On COCO-STUFF, the ADE20K-selected Spatial-SAE configurations improve interpretability measures across all four sparsifiers, while spectral and ablation analyses characterize the reconstruction trade-offs and design choices.

Regularization strength

  • At µ = 10−3, regularization improves Normalized F1 in 100% of matched comparisons and region coherence in 86%, but R2 in none; at 10−1, Normalized F1 falls by a median 43.14 points.The authors select µ = 10−3 for subsequent comparisons, avoiding the steeper reconstruction loss at 10−2.Source: Table 5 · p. 43, §D.4 · pp. 43–44

Code decomposition

  • At µ = 10−3, two-stream decomposition raises Normalized F1 by 6.12±1.86 points on average across all 14 matched comparisons, while retaining a separate channel for high-frequency residual structure.The best two-stream setting reaches 71.0% Normalized F1 at R2 = 35.1%, versus 66.0% at R2 = 41.9% for the best single-stream setting.Source: §D.4 · p. 43

Smooth-field estimator

  • Context attention provides a better trade-off than the static estimator, with median changes of −2.55 points in R2, +1.95 in Normalized F1, and +2.69 in coherence.It is better than the static estimator in 0%, 71%, and 93% of matched comparisons on those metrics, respectively, and is used in the final models.Source: §D.4 · pp. 44–45

Budget selection

  • After fixing context attention and µ = 10−3, the selected kS:kR allocations at total budget 16 are 14:2 for TopK, 12:4 for BatchTopK, 10:6 for ReLU, and 14:2 for JumpReLU.Each allocation is chosen by first retaining settings within 0.5 Normalized F1 points of that sparsifier’s best, then selecting the one with highest R2.Source: §D.4 · pp. 42, 45

Transfer to COCO-STUFF

  • Under direct transfer on COCO-STUFF, SPATIAL-SAE improves Normalized F1 by 1.4–7.1 points and region coherence by 6.5–9.0 points across all four sparsifiers.Map quality and monosemanticity also improve in every comparison by 0.34–0.74 and 0.14–0.73, respectively, on their native 1–10 scales; transferred configurations are retrained from scratch over five random seeds.Source: §D.4 · p. 46

Qualitative feature maps

  • Despite image selection deliberately favoring baseline features, SPATIAL-SAE maps show more complete, connected regions on ADE20K and COCO-STUFF.The examples include extended regions and bounded objects on ADE20K, and objects with varied shapes and scales on COCO-STUFF.Source: Figure 13 · p. 44, Figure 14 · p. 45, §D.4 · p. 46

Spectral results

  • Across four sparsifiers and both datasets, baseline code underallocates low-frequency energy and overallocates high-frequency energy, with upper-spectrum log ratios exceeding 2; SPATIAL-SAE moves the ratios toward zero.The correction persists under tuning-free transfer to COCO-STUFF.Source: §D.4 · p. 46
Loading 2610.04112v1…