PaperlayerView exact revision on arXiv

Source-linked AI summary

Make it SING: Analyzing Semantic Invariants in Classifiers

Harel Yadid, Meir Yossef Levi, Roy Betser, Guy Gilboa

arXiv:2603.14610v2cs.CVeess.IV

TL;DR

Classifier null spaces create equivalent inputs with identical outputs, yet existing methods provide limited human-interpretable insight into their semantic content. SING constructs equivalent images, translates network features into CLIP space, and analyzes the resulting semantic shifts across images, classes, and models. It reports semantic leakage in ResNet50 and stronger preservation of class semantics in DinoViT, while quantitative claims rely on logits and CLIP embeddings.

  • Problem

    Classifier invariants arise from linear-layer null spaces and induce identical outputs, but their semantic content remains difficult to interpret.

  • Method

    SING uses SVD-based null-space projections, equivalent feature pairs, and a learned linear translator from classifier features to CLIP image space for semantic analysis.

  • Results

    SING reveals semantic attributes leaking into ResNet50’s null space, while DinoViT shows the least class-relevant leakage among examined networks and preserves class semantics across invariant space.

  • Takeaways & Limitations

    SING provides human-readable invariant evidence for probing individual images, analyzing classes, comparing architectures, and investigating spurious correlations.

  • Takeaways & Limitations

    The linear translator fits the reported setting empirically, and visualizations are qualitative; quantitative claims rely on logits and CLIP embeddings.

Abstract

from arXiv · show

All classifiers, including state-of-the-art vision models, possess invariants, partially rooted in the geometry of their linear mappings. These invariants, which reside in the null-space of the classifier, induce equivalent sets of inputs that map to identical outputs. The semantic content of these invariants remains vague, as existing approaches struggle to provide human-interpretable information. To address this gap, we present Semantic Interpretation of the Null-space Geometry (SING), a method that constructs equivalent images, with respect to the network, and assigns semantic interpretations to the available variations. We use a mapping from network features to multi-modal vision language models. This allows us to obtain natural language descriptions and visual examples of the induced semantic shifts. SING can be applied to a single image, uncovering local invariants, or to sets of images, allowing a breadth of statistical analysis at the class and model levels. For example, our method reveals that ResNet50 leaks relevant semantic attributes to the null space, whereas DinoViT, a ViT pretrained with self-supervised DINO, is superior in maintaining class semantics across the invariant space.

1. Introduction

SING addresses the difficulty of interpreting semantic information encoded in classifier invariants by connecting null-space geometry to a multimodal vision-language space. It enables human-readable analysis of equivalent inputs across images, classes, and models.

  • Motivation: Classifier null spaces induce equivalent sets of inputs with identical outputs, but their semantic content is difficult to interpret.Nonsemantic invariants can be beneficial, whereas semantic invariants may harm classification; augmentation alone does not reveal what the model learned.
  • SING: SING maps classifier invariant directions into a multimodal vision-language space to provide systematic textual descriptions and visual examples.The method combines classifier geometry with CLIP-based semantic correlations for human-readable interpretation.
  • Scope: The framework supports invariant analysis at the image, class, and model levels, including debugging, comparison, and detection of spurious correlations.It can measure how much a specific concept is ignored by a model and assess semantic leakage into the null space.
  • Model comparison: SING introduces a protocol for comparing architectures by measuring semantic information leaked into their null spaces.The introduction reports that DinoViT had the least class-relevant leakage among examined networks while permitting broad invariants such as background or color.
  • Prior limitations: Existing latent-space SVD methods are data-dependent, while weight-induced null-space methods focus directly on the classifier’s decision geometry.Feature-space axes reflect dataset covariance and may miss invariants residing in the classifier’s null space.

3. Method

SING decomposes classifier features into principal and null subspaces, translates them into CLIP space, and measures semantic changes in equivalent features. It supports null-space analysis through controlled perturbations and image-, class-, attribute-, and model-level diagnostics.

  • 3.1. Setup: SING decomposes the classifier head with SVD, isolates its null-space projector, and treats null-space perturbations as invariant changes that preserve logits.The last fully connected layer maps penultimate features to logits; any perturbation in the null space leaves those logits unchanged.
  • 3.3. Training a translator: A learned linear translator maps classifier features into CLIP image space, enabling semantic interpretation through text-image alignment.The translator is trained to map classifier features to corresponding CLIP image features and supports additive feature decompositions.
  • 3.4. Metrics: Attribute Score measures null-space effects on alignment with a text prompt, while Image Score measures broader semantic change between an image and its equivalent pair.Positive Attribute Score indicates that the equivalent image is closer to the target text; Image Score captures appearance changes that Attribute Score may miss.
  • 3.5. Applications: Quantitative claims rely on logits and CLIP embeddings, whereas synthesized visualizations serve only as qualitative illustrations.This distinction separates the measurement procedure from rendered examples used to display semantic shifts.
  • 3.5. Applications: The framework quantifies semantic leakage at model, class, attribute, and image levels to compare architectures, diagnose classes, and inspect individual failure cases.Model analysis balances a rich invariant space against class preservation; class and attribute analyses use Attribute Score, while image analysis provides fine-grained debugging.
  • 3.5. Applications: Concept-directed perturbations follow a CLIP similarity gradient, project it into the null space, normalize the direction, and apply a controllable step size.Prompts can target another class or attribute, revealing confusing inter-class relationships while keeping the perturbation in the invariant subspace.

4. Experiments

Experiments evaluate SING across models, classes, open-vocabulary concepts, and directed null-space perturbations. DinoViT generally preserves class semantics better, while ResNet50 and other models show greater semantic leakage or steerability.

  • Dataset and models: Five ImageNet-1k models are compared using 10k feature vectors per model across all 1,000 classes.Each model receives a dedicated translator in the same 1,000-class setting.
  • Model comparison: DinoViT achieves the best IS/AS trade-off, while ResNext101 shows high and variable Attribute Score, indicating class-dependent semantic leakage.The model ordering remains unchanged when EVA02 replaces CLIP as the multimodal target.
  • Class analysis: DinoViT typically has |AS| < 1 across classes, whereas ResNet50 exhibits larger and more variable leakage, including Porcupine and Sports-Car.The models’ per-class AS rankings show no significant correlation, suggesting the effect is model-dependent rather than driven by class structure.
  • Open-vocabulary concept analysis: For Arabian Camel and Jellyfish, open-vocabulary analysis compares CLIP angles and Attribute Scores across 30 concepts after null removal.Shorter shifts indicate minimal class-dependent leakage, while small AS for loosely related concepts can indicate spurious correlation.
  • Gradient direction analysis: Null-space perturbations guided toward text prompts shift ResNet50 feature semantics toward target concepts while leaving classifier-head logits unchanged.The demonstrated targets include Arabian Camel, Starfish, Pirate, Jellyfish, and Jeep.
  • Gradient direction analysis: At a fixed null-space step calibrated to IS = 40°, DinoViT has low AS, whereas EfficientNet and ResNet50 show large AS toward a jellyfish prompt.The comparison indicates greater resistance for DinoViT and easier semantic steering for EfficientNet and ResNet50.

5. Discussion and Conclusion

SING constructs logit-equivalent images and uses them to compare semantic invariance across models, classes, and individual images. The analyses show that semantics can drift within the null space even when classifier outputs remain fixed.

  • Discussion and Conclusion: SING generates equivalent images whose logits are identical to those of the original images by construction.The method creates these pairs by perturbing only the invariant component.
  • Discussion and Conclusion: Table 1 compares models under fixed null-space perturbations calibrated to IS = 40°, with DinoViT showing the greatest resistance to directed manipulation.ResNext101 is comparatively susceptible in the reported comparison.
  • Discussion and Conclusion: SING supports model-level sensitivity comparison, class-level identification of semantic shifts, and image-level debugging through human-readable equivalent pairs.These analyses complement accuracy with diagnostics at model, class, and individual-image levels.
  • Discussion and Conclusion: The paper proposes directed augmentation and linear-algebraic control as future ways to move useful semantics from the null space to the principal space while preserving logits.Suggested controls include projector regularization, rank adjustment, and constrained updates.

1. Setup and reproducibility

The translator maps classifier features into CLIP space using cosine-similarity-based training objectives. The reported training behavior indicates that minimizing mean-squared error alone can suffice for high cosine similarity, unlike the reverse.

  • Translator training: Cosine similarity loss is one of three translator-training objectives evaluated with L2 regularization.The translator maps penultimate-layer classifier features to CLIP image features.

3. MSE + Cosine loss

The translator experiments compare MSE and cosine objectives and linear versus nonlinear architectures. MSE optimization achieves high cosine similarity, while nonlinear variants offer no significant advantage over the linear baseline.

  • Minimizing LMSE alone achieves high cosine similarity, whereas optimizing Lcos alone does not reliably reduce MSE.The relationship between the objectives is asymmetric: MSE minimization improves both objectives, but cosine-only training leaves MSE substantially higher.
  • The baseline translator is a linear map selected for stability and compared against three nonlinear translator architectures.The nonlinear models include two multilayer perceptrons and a residual MLP.
  • On a 2,000-image validation subset from 16 classes, no nonlinear architecture shows a significant advantage over the linear translator.The same conclusion is reported in Table 2 and Figure 8.

2. Null space validation

SING validates null-space perturbations by comparing them with equal-norm random and principal perturbations. Null perturbations preserve classifier outputs and global class semantics, unlike the alternatives.

  • Null perturbations lie in the classifier head’s approximate null space, whereas random and principal perturbations provide equal-norm comparison directions.Principal perturbations are selected along directions that strongly affect logits.
  • Null-space perturbations produce negligible logit changes, while random and principal perturbations cause noticeable shifts.The perturbations are matched in ℓ2-norm, and logit changes are summarized over a validation set.
  • UnCLIP generations preserve global class semantics under null-space perturbations but show more noticeable semantic changes under random and principal perturbations.The visualization uses three perturbation types across four random seeds.
  • Figure 10 relates angular distance to perceived semantic change: small angles correspond to nearly identical images, while larger angles indicate stronger changes.The figure presents examples with different attribute and image scores.

3. Image-level and visualization details

SING maps classifier features and their perturbations into CLIP image-embedding space for controlled visual interpretation. Angle-based scores connect embedding changes to perceived semantic differences, while deterministic decoding isolates feature effects.

  • Attribute Score and Image Score are angles in CLIP image-embedding space that quantify semantic changes.Angles below roughly 3° for AS and 10° for IS correspond to barely perceptible changes.
  • UnCLIP combines a text-to-image-embedding prior with a diffusion decoder and super-resolution modules to synthesize images.CLIP’s normalized embeddings make angular similarity the primary carrier of semantic information.
  • A trained translator maps classifier features and perturbed features into CLIP image-embedding space before visualization.The translated perturbation is rescaled to the original feature’s norm to preserve angular relationships and restore the radial component.
  • Fixing the diffusion noise tensor makes outputs deterministic for a fixed CLIP image embedding.The same noise is reused across images and both decoder stages.
  • Different translator architectures produce qualitatively consistent UnCLIP visualizations despite small quantitative cosine-similarity differences.

4. Model-level result extensions

The extended analyses broaden SING’s model-level validation across multimodal targets, architectures, and translator robustness. Class-level violin plots and concept neighborhoods support the consistency and generality of the observed null-space patterns.

  • 4. Model-level result extensions: The extended validation repeats model-ratio comparisons with EVA02 and expands architectural coverage from 5 to 13 ImageNet-pretrained models.The update also evaluates translator robustness using classifier heads trained on principal features before and after translation.
  • 5. Class-level analyses: Null-space perturbations preserve global class semantics in UnCLIP generations across four random seeds.
  • 4. Model-level result extensions: Across architectures, null-space semantic angle changes are summarized as per-class violin distributions, with narrow distributions around zero indicating invariance.The figures cover all participating models and classes.
  • 4. Model-level result extensions: The consistent per-class pattern across architectures with different inductive biases supports the generality of the observations.
  • 5. Class-level analyses: DinoViT’s highlighted concepts are representative of broader open-vocabulary neighborhoods based on CLIP-similarity graphs.Nodes represent text prompts and the target class, while edge strengths reflect CLIP similarity between image and text embeddings.

6. DinoViT feature wrapper

The DinoViT feature wrapper exposes penultimate class-token features and the original classifier head, enabling SING to construct translators and null-space perturbations directly in the model’s feature space.

  • Classification head: It applies the original classifier head with weights W to map the exposed feature f to logits ℓ(f) = Wf.
  • Feature extraction: The SelectClassToken module reshapes token sequences by batch and selects token index 0 as the class representation.
  • Wrapper design: DinoHookable stores the feature extractor, classifier head, and class-token selector while preserving the backbone’s original classification pathway.
  • Feature extraction: The wrapper extracts the penultimate feature f from DinoViT’s encoder.ln output and selects the class token before classification.The class token is selected from the sequence of pre-classifier tokens.
  • Wrapper design: Direct access to this feature space supports construction of linear translators and null-space perturbations for SING’s analysis.
Loading 2603.14610v2…