PaperlayerView exact revision on arXiv

Source-linked AI summary

Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignment

Harrish Thasarathan, Julian Forsyth, Thomas Fel, Matthew Kowal, Konstantinos G. Derpanis

arXiv:2502.03714v2cs.CVcs.LG

TL;DR

Existing interpretability work often studies models separately or aligns concepts after individual-model discovery, while a shared cross-model concept space remains a goal. USAEs jointly encode and reconstruct activations from multiple vision models in one sparse concept space. Experiments report diverse shared concepts, important universal features, model-specific features such as DinoV2’s, and coordinated activation maximization, with sensitivity to model count and a last-layer scope.

  • Problem

    The paper addresses the limited ability of single-model and post-hoc interpretability methods to provide a unified lens on diverse models’ hidden representations.

  • Method

    USAEs jointly encode activations from multiple models into a shared sparse concept space that reconstructs each model’s activations.

  • Results

    Experiments across three vision models discover diverse universal concepts, report a correlation between universality and importance, identify unique DinoV2 features, and reveal representations absent from model-specific SAEs.

  • Takeaways & Limitations

    USAEs provide a framework for interpretable cross-model analysis and coordinated visualization of universal concepts across models.

  • Takeaways & Limitations

    Universal training is sensitive to hyperparameters as more models are added and is limited to discovering features at the last layer of each vision model.

Abstract

from arXiv · show

We present Universal Sparse Autoencoders (USAEs), a framework for uncovering and aligning interpretable concepts spanning multiple pretrained deep neural networks. Unlike existing concept-based interpretability methods, which focus on a single model, USAEs jointly learn a universal concept space that can reconstruct and interpret the internal activations of multiple models at once. Our core insight is to train a single, overcomplete sparse autoencoder (SAE) that ingests activations from any model and decodes them to approximate the activations of any other model under consideration. By optimizing a shared objective, the learned dictionary captures common factors of variation-concepts-across different tasks, architectures, and datasets. We show that USAEs discover semantically coherent and important universal concepts across vision models; ranging from low-level features (e.g., colors and textures) to higher-level structures (e.g., parts and objects). Overall, USAEs provide a powerful new method for interpretable cross-model analysis and offers novel applications, such as coordinated activation maximization, that open avenues for deeper insights in multi-model AI systems

1. Introduction

The paper targets a shared, interpretable concept space for understanding diverse neural networks, addressing the limits of analyzing models independently. USAEs jointly learn and evaluate such a space across vision models, revealing shared and model-specific concepts.

  • Motivation: Existing interpretability methods often analyze one model at a time, making risk management harder as capable neural networks proliferate.The paper motivates a canonical basis for understanding model internals across networks.
  • Motivation: Prior studies suggest differently trained models can share semantically equivalent representations, but universal-feature methods typically discover and match concepts post hoc.This retrospective matching can require compute-intensive filtering or optimization.
  • Approach: USAEs jointly encode activations from multiple DNNs and impose concept alignment during end-to-end training.The framework learns a shared concept space intended to reconstruct and interpret activations across models.
  • Findings: Experiments with three diverse vision models uncover universal concepts spanning low- and high-level abstraction.Reported findings include colors, textures, parts, objects, and other shared concepts.
  • Findings: The study reports a strong correlation between concept universality and importance, alongside unique features in DinoV2.Universal training also reveals shared representations not found through model-specific SAE training.
  • Applications: Coordinated activation maximization visualizes universal concepts simultaneously across models.This application is presented as a way to compare how shared concepts manifest across networks.

2. Related work

Related work covers concept-based interpretability, sparse autoencoders, and studies of shared representations across neural networks. It positions USAEs as a jointly trained alternative to post-hoc discovery and matching across distinct vision architectures.

  • Concept-based interpretability: Concept-based interpretability seeks structured, human-interpretable accounts of internal computations beyond attribution methods.The cited literature frames concept-based approaches as a response to attribution limitations.
  • Sparse autoencoders: Sparse autoencoders use dictionary learning to uncover interpretable concepts in DNN activations.SAEs are presented as a specific instance of dictionary learning.
  • Feature universality: Feature-universality research compares shared information across DNN representations, often using representational alignment and paired data.Recent work also proposes convergence toward shared representations of the world.
  • Universal sparse autoencoders: USAEs randomly select one model-specific encoder to encode activations, then decode the shared concept space toward other models’ activations.The figure describes the forward-pass training process; the supplied text gives only a partial equation and decoder description.
  • Universal sparse autoencoders: Unlike post-hoc approaches that match concepts after individual-model discovery, USAEs learn universal concepts across distinct model architectures for vision tasks.The related-work discussion contrasts retrospective discovery with direct cross-architecture learning.

3. Method

USAEs learn a shared sparse concept code that reconstructs activations from multiple models, aligning their internal representations. The method also supports coordinated activation maximization, visualizing how universal concepts appear across models.

  • Sparse autoencoder background: An SAE maps activations into sparse codes and reconstructs them through a learned overcomplete dictionary.TopK sparsity constrains the code while preserving faithful reconstruction.
  • Universal sparse autoencoders: USAEs extend standard SAEs by learning one shared sparse code that reconstructs every model through model-specific dictionaries.This unified code aligns representations across models while retaining each model’s feature dimension.
  • Training USAEs: Training samples one model’s activations, encodes them into the shared space, reconstructs all models, and aggregates their reconstruction errors.The strategy avoids the memory cost of simultaneously encoding all models while avoiding the slow convergence of sampling model pairs.
  • Training USAEs: A single encoder-decoder pair is updated per iteration, promoting concept alignment while balancing training speed and memory usage.The selected encoder and associated dictionary receive the optimizer update after the universal loss is computed.
  • Coordinated Activation Maximization: Coordinated Activation Maximization independently optimizes one input per model so all inputs maximize the same universal concept dimension.Comparing optimized inputs reveals consistent and divergent manifestations of a shared concept; regularization promotes natural inputs.
  • Coordinated Activation Maximization: Heatmaps visualize universal concepts across models, with bright green indicating stronger activation and examples spanning colors, shapes, parts, objects, and groupings.The qualitative results cover multiple levels of visual abstraction.

4. Experimental Results

Experiments show that USAEs learn interpretable concepts shared across vision models, while also revealing model-specific representations and enabling coordinated visualization. Quantitative analyses support universality, importance, cross-model reconstruction, and partial consistency with independently trained SAEs.

  • Universal concept visualizations: USAE training on three diverse vision models yields robust concepts that generalize across architectures, tasks, datasets, and ImageNet classes.Examples include compositional animal-face concepts that persist even in ViT despite its lack of explicit structured supervision.
  • Activation reconstruction: Positive off-diagonal R2 scores demonstrate successful cross-model activation reconstruction, indicating that USAEs capture shared features.Self-reconstruction is highest on the diagonal; average individual-SAE R2 is 0.8 for DinoV2 versus 0.7 for SigLIP and ViT.
  • Concept universality and importance: Over 1000 concepts reach peak firing entropy, supporting a strongly universal concept space, while a second group activates preferentially for some models.Few concepts occupy the low-entropy range of 0.0–0.2, suggesting most concepts are shared rather than model-specific.
  • Concept universality and importance: The first approximately 100 energetic concepts co-fire over 50–80% of the time, forming a core set of consistently recognized concepts across networks.Co-firing proportions differ by model: 0.266 for DinoV2, 0.344 for SigLIP, and 0.326 for ViT.
  • Concept consistency between USAEs and SAEs: Universal concepts partially overlap with independently trained SAE concepts: 38% for ViT, compared with 23% for SigLIP and 26% for DinoV2.The universal-training AUC exceeds the baseline AUC=0.13, while lower overlap also indicates that universal training discovers concepts absent from independent SAEs.
  • Coordinated Activation Maximization: Coordinated activation maximization visualizes the same universal concept across models and exposes representational differences, such as DinoV2’s larger-scale curves and other models’ animal-sensitive brown-grass activations.Visualized concepts span curves, crosses, textures, round objects, objects viewed from above, and keypads.
  • Unique DinoV2 concepts: DinoV2-specific concepts encode perspective, depth, view invariance, and low-level geometric structure, consistent with its self-distillation and masked-image-modelling objectives.Examples include converging perspective lines, depth-varying object groupings, uphill surfaces, prism faces, dome brim lines, and object corners.

5. Conclusion

USAEs learn a unified concept space that reconstructs and interprets activations across multiple vision models. Experiments find shared and model-specific concepts, plus coordinated activation maximization across models.

  • USAEs learn a unified concept space that faithfully reconstructs and interprets activations from multiple deep vision models simultaneously.
  • The learned concepts range from low-level primitives, including colors, shapes, and textures, to compositional, semantic, and abstract structures such as parts and faces.
  • Many concepts are universal across architectures and training objectives while also accounting for a large proportion of each model’s reconstruction.
  • DinoV2 encodes unique features despite sharing much of its conceptual basis with other models.
  • Universal training recovers many concepts found by independent single-model SAEs and uncovers additional shared representations absent from model-specific training.
  • Coordinated activation maximization enables simultaneous visualization of a universal concept across multiple models.

Impact Statement

Understanding shared representations across DNNs is presented as important for scalable interpretability. The work aims to support risk mitigation, model design, and regulatory compliance through a unified framework for diverse architectures.

  • Understanding shared representations across DNNs is essential for scalable interpretability.
  • A unified framework for diverse architectures is intended to support more effective risk mitigation, robust model design, and compliance with evolving regulations.

A.1. SAE Training Implementation details

The SAE uses an l1 loss and a linear-batch-normalization-ReLU encoder with a dictionary decoder. Experiments use an overcomplete dictionary, standardized activations, and model-specific training adjustments.

  • The SAE replaces the l2 loss with an l1 loss to improve training dynamics and learned-concept interpretability.
  • The encoder combines a single linear layer, batch normalization, and ReLU, while the decoder is a simple dictionary matrix.
  • The dictionary contains 6144 features, an expansion factor of 8 over the largest model feature dimension of 768.
  • Each encoder-decoder pair uses an independent Adam optimizer with an initial learning rate of 3e−4, decaying to 1e−6 through cosine scheduling with linear warmup.
  • Class tokens are removed from DinoV2 and ViT, DinoV2 tokens are interpolated to 16 × 16 patches, and training takes approximately three days on one NVIDIA RTX 6000.

A.2. Unique DinoV2 Concepts

DinoV2 contains unique concepts related to depth, perspective, and geometric cues. Examples include depth cues, corners, rectangular-prism top faces, and dome brims.

  • DinoV2-only concepts relate to depth, perspective, and geometric cues.
  • Concepts 4756 and 1710 fire for depth cues involving foreground objects and backgrounds.
  • DinoV2-independent features include corners, rectangular-prism top faces, and dome brims.

A.3. Unique SigLIP Concepts

SigLIP-specific concepts can respond to both visual forms and their textual counterparts, reflecting its image-language training.

  • Some SigLIP concepts fire for both visual and textual elements representing the same concept.Examples include stars and letters, musical instruments and notes, and the letter C.
  • Concept 5718 responds to star shapes and printed words or partial letters across image regions and scales.
  • These mixed visual-textual activations are consistent with SigLIP’s contrastive objective aligning image and text representations.The visualizations show textual concepts even though USAEs were not trained on language-model activations.

A.4. Out-of-Distribution Generalization

ImageNet-trained USAEs show evidence of out-of-distribution generalization on CelebA and DTD, with consistent reconstruction and co-firing behavior and visualized cross-dataset concepts.

  • ImageNet-trained USAEs show strong evidence of generalization on the out-of-distribution datasets CelebA and DTD.
  • Activation reconstruction remains consistent across the new datasets when measured by MSE and R2.The supplied table caption states that lower MSE and higher R² indicate better performance.
  • The reported concept examples include star shapes and letters, capital C, and music represented through instruments and notes.

A.5. Additional Results

Additional analyses show that universal concepts overlap with independently learned concepts, span low- and high-level visual features, and remain subject to scope and interpretability limitations.

  • Concept consistency: Universal concepts are more consistent with independently trained SAE concepts when their co-firing is higher.Among the top 1,000 co-firing concepts, DinoV2 has average cosine similarity of 0.65, followed by ViT and SigLIP.
  • Qualitative results: Universal concepts include low-level textures such as shells, wood, and tiling.
  • Qualitative results: They also include higher-level concepts involving auditoriums, ground contact, and animal snouts.
  • Zero-shot generalization: The ImageNet-trained USAE shows consistent universality trends and concept-importance relationships on CelebA.
  • Limitations: The method is sensitive to hyperparameters as the number of jointly trained models increases and currently focuses on final-layer features.The authors leave universal-feature analysis across different layer depths for future work.
  • Limitations: A small percentage of discovered concepts remain uninterpretable, potentially because they are stored in superposition or difficult for humans to understand.
Loading 2502.03714v2…