PaperlayerView exact revision on arXiv

Source-linked AI summary

SpIn-ViT: Designing a Sparsity-Induced Vision Transformer That Is Mechanistically Interpretable

Philip H. Lee, Parth Padalkar

arXiv:2608.14922v1cs.CVcs.AI

TL;DR

Post-hoc SAEs in Vision Transformers are trained on frozen representations, leaving patch-level features weakly aligned with classification. SpIn-ViT jointly trains a ViT and modified SAE, outperforming post-hoc SAE baselines across nine benchmarks in accuracy and interpretability while supporting smaller, more accurate neurosymbolic rule sets.

  • Problem

    Patch-level ViT representations lack direct classification supervision and can contain entangled, polysemantic features that are difficult to interpret.

  • Method

    SpIn-ViT jointly trains a pretrained ViT and modified SAE end-to-end to align sparse patch-level latent features with classification.

  • Results

    Across nine benchmarks, SpIn-ViT improves average accuracy by 8.84% over the post-hoc SAE baseline, with nearly 4× higher AI-based and over 2× higher human interpretability scores.

  • Takeaways & Limitations

    Its latent neurons support neurosymbolic models that are 5.97% more accurate with 58.85% smaller rule sets than models built from post-hoc SAE representations.

Abstract

from arXiv · show

Mechanistic interpretability has recently expanded to Vision Transformers (ViTs), with Sparse Autoencoders (SAEs) increasingly used as post-hoc tools to decompose internal representations into sparse and more interpretable features. However, because post-hoc SAEs are trained on frozen representations after the ViT has already been optimized, their latent features are not directly aligned with the downstream classification objective. We introduce SpIn-ViT, a framework that jointly trains a pretrained ViT and a modified SAE end-to-end, directly aligning sparse patch-level representations with image classification. SpIn-ViT learns semantically coherent neuron activations that localize meaningful image regions while maintaining competitive predictive performance. We evaluate SpIn-ViT across nine image-classification benchmarks using classification accuracy, quantitative interpretability metrics, AI-based and Human evaluations. Compared with the previous state-of-the-art post-hoc SAE method, SpIn-ViT achieves 8.84% higher average classification accuracy, an AI-based interpretability score nearly four times as high, and a human-evaluation score more than twice as high. We further extract interpretable rule-sets using the SAE neurons to create neurosymbolic models which achieve 5.97% higher average classification accuracy while requiring a 58.8\% smaller rule-set than the neurosymbolic models created from the SOTA post-hoc SAE method.

Introduction

SpIn-ViT jointly trains a pretrained ViT and modified SAE end-to-end so sparse patch-level features align with classification while remaining interpretable. Across 9 benchmarks, it improves predictive performance, interpretability, and neurosymbolic rule compactness over post-hoc SAE representations.

  • Introduction: Post-hoc SAEs learn latent features independently of downstream classification objectives, so their sparse or coherent features may not capture class-discriminative concepts.ViT patch-level representations also receive no direct supervision when fine-tuning uses only the class token.
  • Introduction: SpIn-ViT jointly trains a pretrained ViT and modified SAE end-to-end, directly aligning sparse patch-level latent features with model predictions.The framework couples sparse representation learning with the classification objective.
  • Introduction: 88.30% average classification accuracy across 9 benchmarks improves over the state-of-the-art post-hoc SAE by 8.84% while remaining competitive with vanilla ViT.SpIn-ViT is evaluated using quantitative, AI-based, and human interpretability measures.
  • Introduction: Nearly 4× higher AI-based and over 2× higher human interpretability scores demonstrate stronger interpretability than the post-hoc SAE baseline.Its latent activations also support symbolic rule extraction through binarization and FOLD-SE-M-induced stratified Answer Set Programs.
  • Introduction: 5.97% higher accuracy and 58.85% smaller rule sets characterize SpIn-ViT’s neurosymbolic models relative to the post-hoc SAE baseline.The rules preserve more classification accuracy while enabling more compact symbolic representations.

Methodology

SpIn-ViT jointly trains a ViT and TopK-based SAE so sparse patch-level features remain predictive while becoming more semantically coherent and interpretable. Its unified objectives combine classification, reconstruction and sparsity, intra-class coherence, and inter-class feature diversity.

  • Architecture Overview: SpIn-ViT applies an SAE to ViT patch-token representations, encoding them into a higher-dimensional sparse latent space and reconstructing the original embeddings.The SAE decoder maps sparse representations back to the ViT embedding space, while the [CLS] token feeds the classification head.
  • Architecture Overview: Unlike post-hoc SAEs trained on frozen features, SpIn-ViT jointly fine-tunes the ViT and SAE with supervision to align sparse representations with classification.This joint training improves interpretability without sacrificing classification performance.
  • Sparse Latent Representation: TopK retains the k largest activations independently for each patch token, producing h ∈ R^n×m with exactly a few active neurons per patch.The latent dimension is m = 10×C, where C is the number of classes; TopK selects active neurons while L1 regularization controls their magnitudes.
  • Learning Objectives: The classification, SAE, center, and orthogonality losses are combined as L = αL_cls + βL_SAE + δL_center + γL_ortho.These components respectively support prediction, information-preserving sparsity, intra-class coherence, and diverse latent features.
  • Learning Objectives: The inter-class objective divides the m-dimensional latent space into C non-overlapping class-indexed blocks and uses orthogonality regularization to reduce redundant dimensions.Each block contains r = m/C latent dimensions, encouraging neurons within a class-indexed subspace to capture diverse visual features.
  • Interpretability Evaluation: After training, interpretability is evaluated by selecting neurons’ strongest reference images, localizing patch-level responses, and measuring prediction relevance and semantic meaningfulness.Neuron activations can also be binarized into tabular data for extracting neurosymbolic rules with FOLD-SE-M.

Neurosymbolic Model Creation

SpIn-ViT’s sparse, classification-aligned SAE neurons are converted into binary predicates for FOLD-SE-M rule induction. The resulting neurosymbolic model combines the ViT-SAE feature extractor with an interpretable ASP rule set grounded in examinable neuron semantics.

  • Motivation and Rule Learning: SpIn-ViT’s sparse, semantically coherent SAE neurons are evaluated for symbolic rule extraction using FOLD-SE-M for consistency with prior frameworks.FOLD-SE-M induces classification rules from labeled tabular data as a stratified Answer Set Program.
  • Rule Learning: FOLD-SE-M’s ratio and tail hyperparameters control the accuracy–complexity trade-off in induced rules.ratio limits false-positive to true-positive coverage in a rule’s default component, while tail sets the minimum number of training examples covered.
  • Rule Extraction from SpIn-ViT: For each image, patch-level SAE activations are averaged into one m-dimensional feature vector before sigmoid transformation and thresholding.The resulting binary activation indicates whether each SAE-neuron predicate is true for the image.
  • Rule Extraction from SpIn-ViT: Each image becomes a tabular row whose binary SAE-neuron activations are inputs and whose ground-truth class is the target for FOLD-SE-M.The induced rules express class predictions through combinations of active neuron predicates, with exceptions represented by alternative predicate combinations.
  • Inference: During inference, the ViT and SAE produce predicate truth values, and the extracted ASP rule set uses them to predict the class.The complete neurosymbolic model consists of the trained neural feature extractor through the SAE latent representation together with the induced rule set.

Experimental Setup and Evaluations

The experiments evaluate SpIn-ViT and baselines across classification, automated and human interpretability, and neurosymbolic performance on nine image-classification benchmarks. Results show that joint training with TopK sparsity yields strong predictive performance and more interpretable, localized features.

  • Experimental scope: Experiments span classification, AI-based and human interpretability, and neurosymbolic model performance across nine benchmark datasets.The datasets include Flowers102, Caltech101, StanfordCars, FGVC Aircraft, EuroSAT, DTD, SUN397, Food101, and OxfordPets.
  • Baselines and variants: SpIn-ViT (Ours) adds TopK sparsity, center loss, and orthogonality objectives to encourage class-aligned, monosemantic features.SpIn-ViT (Vanilla) removes the TopK sparsity constraint, while SpIn-ViT (Ours) includes it.
  • Insertion and Deletion Analysis: 0.830 insertion AUC and 0.142 deletion AUC are achieved by SpIn-ViT (Ours), indicating that its activations identify pixels used for predictions.It rises fastest when highly attributed pixels are inserted and falls fastest when they are deleted.
  • Human Evaluation Results: 4.10 is SpIn-ViT (Ours)’s highest human interpretability score, exceeding SpIn-ViT (Vanilla)’s 3.33 across Flowers102 and FGVC.Ratings came from 12 participants using a 1–5 scale for the coherence of activated visual concepts.
  • AI Evaluation: SpIn-ViT (Ours) ranks highest across all nine datasets in AI-based interpretability, with the ordering closely matching human evaluations.AI evaluation computes mIoU between neuron-highlighted regions and SAM3-generated concept masks, then normalizes scores to a 1–5 scale.
  • Neurosymbolic Model Analysis: SpIn-ViT (Ours) achieves the highest accuracy on all nine datasets and the fewest rules on eight of nine.It trails SpIn-ViT (Frozen) on DTD in rule count despite a 2.4 percent accuracy gain; on Caltech101 and Flowers102 it uses 5 and 10 rules.

Related Works

Prior work uses SAEs to disentangle representations into interpretable features and manipulate model behavior, including in vision models and ViTs. Neurosymbolic vision methods instead convert learned image features into explicit, human-auditable rule-sets.

  • SAE for Mechanistic Interpretability: SAEs disentangle transformer representations into increasingly fine-grained monosemantic features and support downstream activation steering.Applications include style modulation, refusal behavior, and alignment-oriented activation editing.
  • SAE for Mechanistic Interpretability: In vision, SAEs uncover diffusion factors, isolate generative concepts such as texture and shape, and recover meaning-aligned token-level features in ViTs.These applications span denoising trajectories, spatial layout, and sparse decompositions of generative models.
  • Neurosymbolic AI for Vision Models: Neurosymbolic vision methods bind learned image features to explicit, human-auditable rules instead of opaque representations.CNN-based approaches extract logic-based rule-sets from convolutional filter image features, including through binarized kernel features and FOLD-SE-M.

Conclusion

SpIn-ViT integrates sparse autoencoders into the classification objective to learn interpretable patch-level representations without sacrificing competitive accuracy. Across nine benchmarks, it surpasses post-hoc SAE baselines in interpretability evaluations and supports more accurate neurosymbolic models with smaller rule sets.

  • Core contribution: SpIn-ViT integrates sparse autoencoders into the classification objective to learn interpretable patch-level representations while maintaining competitive accuracy.The framework jointly connects sparse representation learning with image classification rather than treating the SAE as a separate post-hoc component.
  • Evaluation: Across nine benchmarks, SpIn-ViT outperforms post-hoc SAE baselines on quantitative, AI-based, and human interpretability evaluations.It also matches or exceeds vanilla ViT performance across the evaluated benchmarks.
  • Neurosymbolic modeling: FOLD-SE-M uses SpIn-ViT’s latent neurons to construct more accurate neurosymbolic models with smaller rule sets than models built from post-hoc SAEs.The latent neurons therefore support compact neurosymbolic model construction in addition to interpretability analysis.

A Appendix · A.1 Training and Evaluation Protocol

The appendix specifies SpIn-ViT’s joint training objective and key hyperparameters, along with FOLD-SE-M rule-extraction settings and the experimental computing infrastructure.

  • A.1 Training and Evaluation Protocol: The overall objective combines classification, SAE reconstruction, center, and orthogonality losses.
  • A.1 Training and Evaluation Protocol: α = 1 and β = 1 weight classification and reconstruction losses equally.
  • A.1 Training and Evaluation Protocol: The sparsity term within LSAE is set to λ = 5e-4.
  • A.1 Training and Evaluation Protocol: For FOLD-SE-M rule extraction, the exception ratio is set to ratio = 1.0.
  • A.1 Training and Evaluation Protocol: The FOLD-SE-M covering limit is set to tail = 5e-4.
  • A.1 Training and Evaluation Protocol: All experiments are run on a single NVIDIA RTX 4090 GPU with 24GB VRAM.

A.2 Further Insertion-Deletion Evaluations and Rule Extraction

This section provides additional evidence beyond the main paper’s averaged results. It reports per-dataset insertion and deletion evaluations and examines how extracted symbolic rules correspond to SAE neuron activations.

  • The section adds per-dataset insertion and deletion results beyond the averaged results reported in the main paper.
  • It also analyzes the correspondence between extracted symbolic rules and SAE neuron activations.

Insertion Results

SpIn-ViT achieves the highest or near-highest insertion AUC on most benchmarks, with especially strong results on Flowers102, Caltech101, EuroSAT, and DTD. Its most salient patches are also the most sufficient for correct classification, as accuracy rises sharply after inserting only 20–30% of pixels and plateaus before baselines.

  • Insertion Results: SpIn-ViT achieves the highest or near-highest insertion AUC on most evaluated benchmarks.The strongest results are reported on Flowers102, Caltech101, EuroSAT, and DTD.
  • Insertion Results: On Flowers102, Caltech101, EuroSAT, and DTD, accuracy rises sharply within the first 20–30% of inserted pixels and plateaus before baselines.This indicates that SpIn-ViT’s most salient patches are also the most sufficient for correct classification.

Deletion Results · Rule Extraction and Correspondence to Soft Patch Segmentation · A.3 Example Labelled Rulesets

SpIn-ViT’s sparse representations identify classification-critical regions and support symbolic rules directly grounded in SAE features. Across benchmark examples, these neurons correspond to localized visual concepts and readable labelled rules.

  • Deletion Results: SpIn-ViT shows the steepest early accuracy decline and lowest deletion AUC on most datasets, indicating its marked regions are necessary for classification.The effect is particularly pronounced on Flowers102, Caltech101, EuroSAT, and DTD.
  • Deletion Results: Model separation is smallest on FGVC, Sun397, and Food101, where classification depends on broader and less concentrated regions.SpIn-ViT (Frozen) and PatchSAE degrade more gradually, consistent with weaker insertion performance and more diffuse attributions.
  • Rule Extraction and Correspondence to Soft Patch Segmentation: FOLD-SE-M rules operate directly in the SAE feature space, with each predicate corresponding to one sparse code dimension rather than a post-training explanation layer.Rules express logical conditions over active SAE neurons in readable form.
  • Rule Extraction and Correspondence to Soft Patch Segmentation: Neuron-induced soft patch segmentations align with symbolic rules: neuron_63 marks pickup-truck bodies and cabs, while neuron_123 marks dog heads and bodies across varied appearances.The corresponding FOLD-SE-M predicates map directly to the visualized neurons.
  • A.3 Example Labelled Rulesets: OxfordPet rules combine positive and negative neuron predicates to distinguish Dog, Cat, Beagle, Persian, and Labrador labels.Examples include label(X,’Dog’) :- neuron_123(X) and label(X,’Beagle’) :- neuron_123(X), neuron_31(X).
Loading 2608.14922v1…