Source-linked AI summary
A Deeper Analysis of Block-Sparse Featurizers
Alexandru-Iulius Jerpelea, Amith Ananthram
TL;DR
Some model features lie on low-dimensional manifolds, limiting standard SAE representations. This paper studies block-sparse featurizers, tests their behavior, and proposes architectural extensions. BSFs recover manifold features as single units but retain feature splitting; Tournament Top-K reduces much of the splitting, while varied block dimensions fail and crosscoder blocks introduce merging.
Problem
Standard SAEs assume linear feature directions, although some features occupy low-dimensional manifolds relevant to interpreting model activations.
Method
The paper studies BSFs on synthetic and real-model settings, analyzes failure modes, proposes Tournament Top-K and varied block dimensions, and extends blocks to crosscoders.
Results
BSFs recover manifold features as single units, but feature splitting persists; Tournament Top-K reduces much of the splitting, while crosscoder blocks create a merging problem.
Takeaways & Limitations
The results support developing featurizers whose atomic units better match the geometry of the data-generating process.
Takeaways & Limitations
Varied block dimensions fail because features lock into wrong-sized blocks early in training, under the assumption that concepts occupy low-dimensional subspaces.
Abstract
from arXiv · showhide
The recently introduced block-sparse featurizer (BSF; Fel et al., 2026) is similar to a sparse autoencoder (SAE), but its atomic unit is a small subspace (a block of directions) rather than a single direction. It is designed for features that live on low-dimensional manifolds, which are especially frequent in vision. This work studies the BSF's strengths and weaknesses, finding how it still somewhat suffers from classic SAE failure modes, like feature splitting and composition. We propose several architectural changes to the BSF, including a Tournament Top-K selection rule that significantly reduces feature splitting, and we also extend the block paradigm to the crosscoder.
1 Introduction
The paper motivates block-sparse featurizers by showing that some model features occupy low-dimensional manifolds rather than linear directions. It studies BSF behavior and proposes extensions and changes targeting its remaining failure modes.
- Motivation: Some model features live on low-dimensional manifolds, challenging the assumption that features are linear directions in activation space.Examples include days of the week arranged on a circle.
- Motivation: SAEs can tile curved manifolds by spending many directions on local regions, producing confusing outputs.The paper reports unsuccessful attempts to recover concept manifolds post-hoc from trained SAEs.
- Contributions: The paper trains BSFs on a toy manifold dataset and tests seed stability, resistance to spurious correlations, and dominance over classical SAEs.
- Contributions: BSFs still tile manifolds, so the paper proposes Tournament Top-K as a response to feature splitting.
- Contributions: The paper proposes BSFs with non-constant block dimensions and reports brief results from training BSFs on real models.
- Contributions: The block paradigm is extended to crosscoders, and the paper discusses whether BSFs can capture geometries beyond low-dimensional manifolds.
2 The BSF
A BSF represents concepts as small latent blocks corresponding to low-dimensional subspaces, applying sparsity at the block level. Its top-k variant selects whole blocks by norm, yielding sparse concept selection but dense representation within each selected concept.
- BSF design: BSFs split their code into small groups of latents called blocks and apply the sparsity penalty to whole blocks.
- BSF design: The underlying hypothesis assigns each concept an orthonormal low-dimensional subspace, with activations formed as sparse sums of concept contributions.Each concept's position within its manifold is represented by z_g.
- BSF design: The studied top-k Grassmannian variant ties the encoder to the decoder transpose and learns only the orthonormal frames.
- BSF design: Top-k selection keeps the k blocks with largest norm, making selection sparse across concepts but dense within each selected block.A whole color circle can switch on as one atom, which should theoretically reduce manifold tiling.
3 The Manifold Zoo
The Manifold Zoo evaluates BSFs on synthetic features with varied geometries, testing recovery, seed consistency, correlated activations, and splitting. BSFs outperform parameter-matched SAEs in correlated settings but still split some manifolds across blocks, even under favorable capacity and sparsity choices.
- 3 The Manifold Zoo: The Manifold Zoo contains 128 embedded features—64 segments and 64 curved manifolds—with each sample summing four concepts.The dataset has 300,000 samples, and each feature is embedded in a 128-dimensional activation space using a random orthonormal frame.
- 3.1 BSF vs. Classic SAE: BSFs outperform parameter-matched SAEs for mean recovery R2, while post-hoc manifold recovery makes SAEs reasonably competitive on independent data.With correlated concept activations, BSF accuracy remains unchanged while SAE accuracy decreases because correlation-based grouping can merge concepts.
- 3.2 BSF Consistency Across Seeds: Segments, spheres, helices, and tori are almost always recovered by one block, whereas other manifolds flicker across the ten random seeds.Every concept is recovered by exactly one block in at least one seed, indicating that no feature is intrinsically impossible to recover.
- 3.2 BSF Consistency Across Seeds: Block firing correlations are close to an identity across seeds, but this alignment fades for features that split across multiple blocks.The correlated dataset produces similar R2 scores and feature-splitting patterns, showing that the seed behavior persists under concept co-firing.
- 3.4 So Why Do Splits Form?: Feature splitting persists across G ∈ {128, 256, 512}, including G = M = 128, and increasing top-k from 4 to 6 or 8 worsens splitting.At k = 8, no feature is captured by one block; the baseline uses G = 256 blocks of dimension b = 3 with top-k = 4.
- 3.4 So Why Do Splits Form?: When manifolds split, the blocks tend to become near-orthogonal complements rather than separate regions of the manifold.The two blocks specialize around the same time, initially develop similar subspaces, then pull apart toward near-orthogonal components.
- 3.4 So Why Do Splits Form?: BSFs split manifolds less often than classical SAEs but still exhibit the same qualitative failure mode, whose cause remains incompletely understood.The authors use the observed evolution of splitting block pairs to motivate a subsequent BSF architecture.
4 A Solution to Feature Splitting: Tournament Top-K
Feature splitting persists even without correlated features: blocks can become near-orthogonal complements, while Tournament Top-K uses their geometry to reduce splitting and improve recovery.
- Tournament Top-K: Tournament Top-K orders blocks by norm, skips geometrically overlapping candidates, and stops after accepting k blocks.A candidate is skipped when its overlap with an accepted block exceeds 0.1.
- Tournament Top-K: The 0.1 overlap threshold is reversible, while overlap above 0.5 creates a permanent winner based on duel wins.Reversibility accommodates early pairs that later specialize to different features.
- Results: Higher sparsity worsens performance; at k = 8, every feature splits across multiple blocks.Training dynamics are tracked through subspace overlap and block-feature firing correlations.
- Why splitting occurs: Feature splitting occurs even on independent data, with split blocks tending toward near-orthogonal complements.The phenomenon resembles manifold tiling by classical SAE directions.
5 Varied-Dimension Blocks
Mixed-dimensional blocks fail to self-sort: selection rules favor blocks of particular sizes, and early assignments become difficult for gradient descent to change.
- Setup: Features have varied dimensionalities, but a mixed-block BSF with 128 1D, 38 2D, and 90 3D blocks fails across tested regimes.The authors observed feature merging, including up to three linear features explained by one block.
- Selection regimes: Vanilla top-K favors 3D blocks because they have higher average norms, causing segments to use 3D blocks while curved features collapse.This is a size-dependent selection bias.
- Selection regimes: Normalized top-K reverses the bias by disproportionately preferring 1D blocks.It divides each block magnitude by its expected scale before selection.
- Selection regimes: MDL scoring and JumpReLU with an L0 penalty also fail to produce the desired allocation of features to block sizes.The mixed-size model can achieve lower loss under optimal assignments, but training does not reach them.
- Training dynamics: Once a feature attaches to a block early, moving it raises loss for a significant time, making reassignment difficult for gradient descent.The authors suggest interventions like Tournament Top-K but leave this unresolved.
6 Applying BSFs on Real Models
Real-model experiments apply BSFs to model comparison, synthetic visual features, causal interventions, and layer maps, but these applications are secondary to studying BSFs themselves.
- Experimental setup: The authors train real-model BSFs on DINOv3 and CLIP using 16,384-dimensional dictionaries, top-K = 64, and block dimension 3.Matryoshka SAEs are trained at the same scale for comparison.
- Applications: Cross-model matching pairs CLIP and DINO blocks by activation correlation, yielding a few hundred strongly matched blocks and few truly private blocks.This experiment probes shared and model-specific representations.
- Applications: Synthetic images varying color, position, orientation, frequency, and shape produce dedicated blocks carrying each feature.The experiment uses artificial visual features.
- Causality: Blocks are causal to their features on ImageNet, but observing this often requires ablating multiple blocks because features are split.The authors ablate and inject blocks on labeled external data.
- Scope: The real-model applications are briefly treated because the paper’s primary focus is the BSF itself.Many experiments are deferred to a follow-up model-diffing paper.
7 Block Crosscoders
Block crosscoders extend block representations to paired models but introduce within-block collisions between model-exclusive features; dedicated feature columns substantially reduce these collisions.
- Construction: The block crosscoder encodes paired hidden states with top-K selection by the largest block norms and decodes through two dictionaries.The same top-K method is maintained for paired samples from models A and B.
- Merging problem: A block crosscoder can assign an A-only feature and a B-only feature to the same block on a Manifold Zoo with 64 shared and 32 exclusive features per model.This is identified as a structural problem of the block extension.
- Merging problem: The collision occurs because different dimensions within a b-dimensional block can independently serve the A-only and B-only features.When their combined dimensionality is ≤ b, both concepts can remain independent within one block.
- Dedicated feature columns: Dedicated feature columns pre-partition blocks into shared, A-only, and B-only pools, making feature collisions much rarer on the toy dataset.The corresponding dictionary is zeroed on the opposite model’s exclusive pool.
- Real-model results: A DINO/CLIP block-DFC crosscoder achieves EVs of 0.71 for DINO and 0.66 for CLIP, with only 1 dead block.The configuration uses 8192 blocks divided among shared, DINO-only, and CLIP-only pools.
- Real-model results: The crosscoder appears to learn DINO’s dictionary and exclusive pool, while treating CLIP as mostly shared and covering it poorly.Whether CLIP features are a subset of DINO’s or simply harder to recover remains open.
8 Conclusion
The BSF recovers manifold-shaped features as single units rather than SAE-like fragments, but retains several classic failure modes. Tournament Top-K and dedicated feature columns mitigate splitting and merging, while mixed block dimensionalities remain problematic.
- BSF recovers manifold features as single units, whereas regular SAEs tile them into fragments.
- Features still split across blocks even without correlations, but split blocks form near-orthogonal complements.
- Tournament Top-K exploits split-block geometry to recover much of what vanilla top-k loses.
- Mixing block dimensionalities fails because features lock into incorrectly sized blocks early in training.
- Block crosscoders introduce a merging problem that dedicated feature columns partly address.