Source-linked AI summary
Do vision models know that three is three?
Ankit Sonthalia, Arnas Uselis, Seong Joon Oh
TL;DR
The paper asks whether vision models encode concept changes consistently across visually unrelated image regimes, a question left open by prior work on vision linearity. It studies concept directions across regimes in 46 vision encoders. Local linearity is nearly universal, but cross-regime alignment is partial and tends to be stronger in models with stronger local linearity.
Problem
Whether concept directions transfer across visually unrelated regimes remains insufficiently studied, despite prior evidence of linear structure in vision embeddings.
Source: §1 · p. 1Method
The authors study 46 vision encoders across architectures, training objectives, model sizes, and pretraining data, examining concepts within and across regimes.
Source: §1 · p. 2, §3.1.1 · p. 4Results
Takeaways & Limitations
Stronger local linearity tends to accompany stronger global alignment, and globally aligned concept directions can serve as linear probes across regimes.
Source: §1 · p. 2, §5 · p. 9The conclusions are limited to the investigated concept types, and counterfactual pairs are approximate, matched by embedding-space distance.
Source: §5 · p. 9
Abstract
from arXiv · showhide
Do vision models encode the transition $2{\to}3$ the same way whether it appears in handwritten digits or in hand gestures, and do they "understand" that clean${\to}$dirty is the same transformation regardless of whether it is applied to a kitchen or a carpet? The linear representation hypothesis (LRH) answers this for language models: concept changes correspond to directions in (language) embedding space. In vision, the same concept can manifest in visually unrelated "regimes" that rarely co-occur: images of digits and hands do not share much in common in pixel space. To understand whether vision models still encode the underlying abstract concepts, we study 46 vision encoders across architectures, training objectives, and scales, on 10 concept families including numerosity, age, object identity, and MIT-States attributes. Within each regime, we find statistically significant concept directions near-universally across our tested models. However, two distinct regimes do not always share the same concept direction. Several models learn multiple linear structures for the same concept transition, split across regimes. We call this phenomenon "fragmented linearity": strong local linearity coexisting with near-absent global linearity. Fragmentation is, however, not fixed: models with stronger local linearity progressively escape it (local-global linearity correlated with $\tau \in [0.38, 0.89]$ across all concepts), and where global linearity does emerge, concept directions can transfer as linear probes across regimes.
1 Introduction
The paper asks whether vision embeddings represent the same abstract concept transition consistently across visually unrelated regimes. Across studied encoders, local concept directions are common, but cross-regime alignment is partial and varies with local linearity.
1 Introduction
- Prior work suggests linear structure in vision embeddings, but whether it survives across visually unrelated regimes remains insufficiently studied.This matters because vision encoders increasingly serve as perceptual backbones for multimodal systems.Source: §1 · p. 1
- The study examines 46 vision encoders spanning architectures, training objectives, and model sizes, on numerosity, age, object identity, and MIT-States attributes.The passage also mentions pretraining data as a dimension of variation.Source: §1 · p. 1
- Within a regime, concepts are almost universally encoded linearly beyond a random baseline, but this structure only partially transfers across regimes.The paper names within-regime structure local linearity, cross-regime structure global linearity, and local structure without transfer fragmented linearity.Source: §1 · p. 2
- Stronger local linearity in CLIP-like models tends to accompany stronger global linearity, while supervised models can show complete fragmentation.The authors suggest regime-specific representations gradually converge toward regime-invariant ones.Source: §1 · p. 2
2 Preliminaries and related work
The paper distinguishes directional linearity from separability and additive decomposability, then situates its study around whether concept directions transfer across visually different regimes. It focuses on the geometry of general vision encoders rather than interpretability methods or representation convergence across models.
2.1 Different notions of linearity
- Linear separability can decode concept classes, but does not ensure consistent directions suitable for representation-space manipulation.The paper distinguishes this from the LRH, which treats concept changes as directions in embedding space.Source: §2.1 · pp. 2–3
- Additive linearity describes primitive concept vectors combining to form composite concepts, a distinct notion from the LRH studied here.Prior work showed approximate additive linearity after tangent-space projection, leaving Euclidean LRH and cross-regime transfer unresolved.Source: §2.1 · p. 3
2.2 Related work
- The study characterizes the geometry of vision embeddings, in contrast to prior interpretability work that seeks to recover concept directions or studies other VLM embedding properties.Its perspective on convergence concerns alignment of concepts across regimes within one model, not a shared canonical representation across models.Source: §2.2 · pp. 3–4
- Unlike prior studies treating linearity as a single global property, this work examines regimes separately and hypothesizes strong local representations with weaker cross-regime transfer.The paper notes an analogy to learning different mechanisms across contexts, while excluding mechanistic interpretability from its scope.Source: §2.2 · p. 3
3 Fragmented Linearity
This section introduces the paper’s definitions of local linearity and fragmentation and establishes the measurements used in the rest of the paper.
3 Fragmented Linearity
- The section defines local linearity and fragmentation and lays out the measurements used throughout the paper.Source: §3 · p. 4
3.1 Regimes and approximate counterfactuals
The study partitions visually distinct data subsets into regimes to test whether concept directions hold within and across them. Because images rarely match on every other concept, it constructs approximate counterfactual pairs within each regime.
3.1.1 Regimes and local linearity
- A regime is a subset of the concept space, allowing linearity to be studied within and across visually different data subsets.The metric follows the LRH definition of linearity.Source: §3.1.1 · p. 4, §3.1 · p. 4
- Local linearity means the chosen linearity property holds inside each subset of the regime partition.Source: §3.1.1 · p. 4
3.1.2 Counterfactual construction
- Approximate counterfactuals are formed by matching embeddings for concept values a and b within a fixed regime.The closest available pairs rely on the assumption that embedding distance reflects similarity in non-target concepts.Source: §3.1.2 · p. 4
3.2 Metrics: direction consistency and fragmentation
The paper measures concept-direction consistency within a regime and across regimes, testing scores against random-pair baselines. It defines fragmentation by how often cross-regime consistency cannot be distinguished from that baseline.
3.2.1 Direction consistency
- Direction consistency (DC) measures whether counterfactual differences align with a common concept direction; high local DC across regimes indicates local linearity.Source: §3.2.1 · p. 5
- Global linearity is summarized by averaging cross-regime DC across distinct regime pairs, excluding identical regimes.High cross-regime DC across all regime pairs implies high global linearity.Source: §3.2.1 · p. 5
3.2.2 Random baseline
- DC is tested against a random baseline built from uniformly sampled pairs without counterfactual matching, using one-sided Welch tests with Bonferroni correction.Source: §3.2.2 · pp. 5–6
- Figure 2 shows supervised models with local but selective global linearity, while CLIP models are significant on most cross-regime pairs.Values indistinguishable from baseline are excluded from display.Source: Figure 2 · p. 6
3.2.3 Fragmentation
- A concept representation is fragmented when cross-regime directions do not transfer consistently; its score is the fraction of regime pairs indistinguishable from baseline.A score of 0 indicates consistency across all pairs, while 1 indicates fragmentation across all pairs under the tested sample sizes.Source: §3.2.3 · p. 6
3.3 Experimental setup
The experiments compare 46 vision models across training objectives and architectures on 10 concept families. The setup includes multiple datasets and visual regimes for concepts such as numerosity, age, object identity, and MIT-States attributes.
3.3 Experimental setup
- The study covers 46 models trained with contrastive, DINO, masked-image-modeling, or supervised objectives, using both CNN and transformer architectures.Source: §3.3 · p. 6
- Table 1 reports fractions of model-regime pairs distinguishable from baseline: local linearity is near-universal, global linearity less common, and the two correlate positively for every concept.Source: Table 1 · p. 7
- The evaluation spans 10 concept families, including numerosity, age, object identity, and seven binary MIT-States attributes.Numerosity uses four datasets; age uses two, and PACS supplies object identity.Source: §3.3 · p. 7
4 Results
Across 46 vision models and 10 concepts, local linearity is nearly universal, but cross-regime linearity is less consistent. Global linearity varies with local linearity, and concept directions can transfer across regimes as linear probes.
4 Results
- The analysis evaluates 46 models across 10 concepts by measuring linearity within and between image regimes.For each model–concept pair, regime-pair DC scores form a matrix whose diagonal entries measure local linearity and off-diagonal entries measure global linearity.Source: §4 · p. 7
- 99.1% of model–concept–regime tests show statistically significant local linearity, versus 52.2% of model–concept–regime-pair tests for cross-regime linearity.The local result covers 2,530 tests; the cross-regime result covers 13,984 tests.Source: §4 · p. 7
- For numerosity, supervised ViT has fragmentation 0.67 while CLIP has 0; for cleanliness, their fragmentation scores are 0.92 and 0.08, respectively.For numerosity, 8 of 12 supervised ViT regime pairs are near baseline, while CLIP’s cross-regime results are all significant; for cleanliness, 11 of 12 supervised ViT pairs are near baseline.Source: §4 · p. 7
- Global linearity passes 99.3% of significance tests for object identity and 94.6% for age, where visual cues remain consistent across regimes.The text contrasts these with abstract concepts such as numerosity and cleanliness, whose regimes differ in subject matter.Source: §4 · pp. 7–8
- Across all concepts, global and local linearity correlate positively across tested models (Kendall τ ∈ [0.38, 0.89], all p < 10^-3); this describes an emerging trend, not a causal effect.Source: §4 · p. 8
- For three representative transitions, probe accuracy correlates positively with global linearity (τ = 0.77, 0.66, 0.53), supporting the semantic usefulness of directions in high-global-linearity models.The probe uses a concept direction estimated in a source regime to classify held-out samples from another regime.Source: §4 · p. 8, Figure 4 · p. 9
5 Conclusion
Vision encoders often represent concepts linearly within a visual regime, but many learn separate structures across regimes; stronger local linearity tends to accompany stronger global linearity. The conclusions are limited to the investigated concept types, and the counterfactual pairs are approximate.
5 Conclusion
- Stronger local linearity tends to accompany stronger global linearity, while many models learn the same concept separately across regimes.Where global linearity emerges, concept directions can act as effective linear probes across regimes; whether regime-specific skills also converge remains open.Source: §5 · p. 9
- The conclusions are limited to the investigated concept types, and counterfactual pairs are approximate because they are matched by embedding-space distance.The authors argue that statistical tests and probing results together provide sufficient evidence for their main claims within this scope.Source: §5 · p. 9
- Linear directions could be exploited to steer representations toward harmful outputs, but the work introduces no attack surfaces beyond those already present in existing models.Source: §5 · p. 9
A Models
Table 2 provides the full list of models used in the study. OpenCLIP entries include model names and pretrained tags, while timm entries use the model names passed to its creation function.
A Models
- Table 2 is the study's full model inventory, with model identifiers reported in the format used by each model library.OpenCLIP entries include the pretrained tag; timm entries use the model name passed to timm.create_model.Source: Table 2 · p. 14, A Models · p. 14
- Other listed timm entries include ResNet-18, ResNet-50, ResNet-101, EfficientNet-B0, and EfficientNet-B4.Source: A Models · p. 14
- The inventory also includes ConvNeXt-B, ConvNeXt-L, ConvNeXt-XL, ViT-S/16, ViT-B/16, and ViT-L/16.Source: A Models · pp. 14–15
- The listed timm entries include TinyViT-21M, DeiT-B/16, Swin-T, and Swin-B.Source: A Models · p. 15