Geometric Stability: The Missing Axis of Representations

Motivation

• Similarity methods answer whether two systems encode comparable pairwise structure but cannot determine whether one system’s geometry remains recoverable across feature subsets.Their invariance to basis transformations makes representations with redundantly or non-redundantly distributed geometry score identically, despite different reliability for probes and interventions.Source: §1 · pp. 2–3

Implications

• Geometric stability is a distinct, substrate-independent axis of representational structure with implications for model selection, training objectives, and reliable mechanistic interpretability.The motivating interventions include linear probes, activation patching, and steering, which assume that targeted geometric structure is consistent across feature subsets.Source: §1 · pp. 2–4

Method

• Shesha measures single-representation geometric stability by correlating RDMs from complementary random feature subsets, rather than comparing alignment between two representational spaces.It averages Spearman rank correlations across K independent splits and is deliberately non-invariant to orthogonal feature transformations, making it sensitive to basis-level redundancy.Source: §1 · p. 3

Empirical validation

• Across 2,463 encoder configurations, stability and similarity correlate positively under geometry-preserving transformations (ρ = +0.75) but negatively under compression (ρ = −0.47).The pooled correlation is near zero (Spearman ρ = −0.01, 95% CI [−0.06, +0.03]), because the transformation regime determines the relationship.Source: §1 · pp. 3–4
• DINOv2 ranks among the two most transferable models on three of six datasets yet falls in the bottom stability quartile on five, an isolated dissociation.Across 170 vision models, transferability and stability are not generally traded off, while contrastively aligned models such as CLIP achieve both high transfer and high stability.Source: §1 · p. 4

Mechanism

• Removing the top principal component collapses CKA to 0.27 while Shesha remains 0.95, whereas retaining only top components recovers CKA but drives Shesha below zero.The double dissociation shows that CKA follows leading components, while Shesha responds to geometry distributed across the coordinate basis.Source: §1 · p. 4

2.1 Formal Definition

• SheshaFS averages Spearman correlations between cosine-distance RDMs computed from complementary random feature halves, with S(X)≈1 indicating recoverable geometry and S(X)≈0 indicating unreliable recovery.The estimator uses K=30 independent partitions and lies in [-1,1]. Spearman correlation makes it invariant to monotonic distance transformations, while cosine distance makes it invariant to global scaling.Source: §2.1 · p. 5, §2.3.5 · p. 7

2.2 Basis Interpretation

• Shesha measures how geometric information is distributed across coordinate axes, jointly determined by eigenspectrum and orientation rather than by the eigenspectrum alone.Redundant variance across coordinates yields high stability, whereas asymmetric concentration across coordinates yields low stability; orthogonal rotations preserve eigenvalues but can change S.Source: §2.2 · p. 5

2.3 Formal Dissociation from Similarity Metrics

• Shesha’s non-invariance to orthogonal transformations is the algebraic mechanism that captures geometric properties invisible to CKA and Procrustes.CKA depends only on XX^⊤, while Procrustes explicitly optimizes over orthogonal matrices; Shesha instead tracks basis redistributions.Source: §2.2 · p. 6, §2.3 · p. 6
• Shesha is invariant to global or isotropic scaling, coordinate permutation, and strictly monotonic distance transformations, while CKA shares scaling and permutation invariance but not rank-based distance invariance.Permutation invariance follows from exchangeability of uniformly sampled partitions and preservation of XX^⊤ by permutation matrices.Source: §2.3.1 · p. 6, §2.3.2 · p. 6, §2.3.5 · p. 7

2.3.3 PCA Compression

• PCA compression preserves dominant variance and approximately preserves CKA, but sharply lowers Shesha by concentrating geometric information into a coordinate subset.This basis configuration differs from DINOv2’s later low stability, which reflects nonredundant variance spread across many coordinates.Source: §2.3.3 · p. 6

2.3.4 Orthogonal Rotation

• Rotating a representation into its eigenbasis collapses SheshaFS from 0.903 to near zero while preserving CKA at 1.000, proving their formal dissociation.Orthogonal rotations preserve XX^⊤, rank, and the eigenspectrum, but redistribute geometric information across coordinates and change SheshaFS.Source: §2.3 · p. 6, §2.3.4 · p. 7

2.4 Connection to the RSA Noise Ceiling

• Unlike the RSA noise ceiling, which audits measurement reliability across observations, Shesha audits architectural geometric reliability across feature dimensions.Both use split-half self-consistency, but along different axes: observations for the noise ceiling and features for Shesha.Source: §2.4 · p. 7

3 Distinctness of Stability and Similarity

• CKA is unchanged by orthogonal rotation because XX⊤ is preserved, whereas Shesha changes as rotation redistributes geometry across coordinate axes, providing the formal dissociation.Global scaling and feature permutation leave both metrics invariant, while PCA compression and rotation expose their complementary blind spots.Source: §3 · pp. 7–8

3.1 Construct Validation

• Synthetic validation recovers controlled stability with near-perfect fidelity (ρ = 0.997, p < 10−86), while cases with CKA > 0.97 and near-zero Shesha establish that similarity and stability are separable.Balanced sampling covers all four stability–similarity quadrants, including adversarial high-similarity, low-stability representations.Source: §3.1 · pp. 8–9

3.2 CKA Tracks Dominant Variance & Shesha Tracks Full-Manifold Geometry

• Removing the leading principal component drops CKA from 1.0 to 0.27 while Shesha remains 0.95, demonstrating that similarity metrics track dominant variance whereas Shesha tracks distributed geometry.At 26 removed components, Shesha retains roughly 92× CKA’s signal; Procrustes and PWCCA collapse identically to CKA, confirming a family-level blind spot.Source: §3.2 · p. 10
• PCA compression restores CKA to 0.972 in vision and 0.993 in language while Shesha falls to −0.136 and −0.098, respectively, with opposite monotonicity across all seven domains.Retained dimension correlates positively with CKA but negatively with Shesha in vision, language, and audio, because PCA fragments split-half geometry across decorrelated, variance-ranked components.Source: §3.2 · p. 10
• Across 2,463 encoder configurations in seven domains, geometry-preserving transformations couple CKA and Shesha positively (ρ = +0.75), whereas PCA compression uniquely produces negative coupling (ρ = −0.47).Natural encoders show weak positive coupling (ρ = +0.33), with Shesha contributing roughly 90% unique variance beyond CKA; encoder identity explains under 10% of stability variance.Source: §3.2 · pp. 10–11

3.3 Geometric Stability Extends to Biological Representations

• Biological representations preserve the dissociation: protein encoders show ρ = −0.36, single-cell profiles ρ = +0.06, and neural recordings ρ = +0.01 across 846 configurations.The pattern spans systems shaped by gradient descent, evolution, development, and biological learning, supporting substrate-independent geometric stability.Source: §3.3 · p. 11

4 Geometric Stability in Pretrained Vision Models

• SheshaFS is highly reproducible across feature-partition seeds, with Spearman ρ ≥0.993 and median per-model coefficient of variation of 0.75%.The full CIFAR-10 sweep was recomputed across three seeds for all 170 models.Source: §4 · p. 12

4.1 The DINOv2 Paradox

• DINOv2 ranks among the most transferable families yet last or near-last in stability on five of six datasets, with EuroSAT as the exception.It ranks first or second in transferability on Flowers-102, CIFAR-10, and CIFAR-100, but 35th–36th in stability there and bottom-quartile on Oxford Pets and DTD.Source: §4.1 · pp. 12–13, Table 2 · p. 13

4.2 DINOv3 Closes the Dissociation

• DINOv3 raises mean SheshaFS from 0.485 to 0.722 versus DINOv2 while lowering mean LogME from 1.286 to 0.976, reversing stability’s scaling collapse.At matched large scale, DINOv3 gains stability with preserved transfer on four of six datasets, while Flowers-102 and Oxford Pets incur transfer costs.Source: Table 2 · p. 13, §4.2 · pp. 14–15, Figure 4 · p. 14

4.3 Architectural and Training Determinants

• Contrastive alignment predicts higher stability across all six datasets, while hierarchical architectures help significantly only on Flowers-102; cross-dataset rank consistency is ρ = 0.95.The alignment target, rather than the training mechanism, determines stability, and multi-scale benefits depend on visual domain.Source: §4.3 · p. 15

4.4 Geometric Stability is Not Effective Dimensionality

• SheshaFS is distinct from effective dimensionality: across datasets it anti-correlates with participation ratio (mean ρ = −0.36) and positively correlates with top-eigenvalue variance share.DINOv2 has the benchmark’s highest participation ratio yet lowest stability on natural-image datasets, because stability depends on variance distribution across learned coordinates.Source: §4.4 · pp. 15–16

4.5 Geometric Stability is Distinct From Corruption Robustness

• Clean geometric stability does not consistently predict corruption robustness: partial ρ is −0.03 on CIFAR-100 and +0.26 on CIFAR-10.Thus SheshaFS diagnoses recoverability from random coordinate subsets, not general representation quality or performance under input distribution shift.Source: §4.5 · p. 16

4.6 Optimizer Geometry Modulates Stability

• SAM holds accuracy nearly constant and steadily lowers CKA while increasing SheshaFS from 0.806 to 0.872 on CIFAR-10 and from 0.805 to 0.822 on CIFAR-100.The CIFAR-10 stability optimum occurs near ρ = 0.05–0.1, whereas CIFAR-100 stability rises across the sweep, demonstrating geometry independent of accuracy and similarity.Source: §4.6 · pp. 16–17, Table 3 · p. 17

4.7 Geometric Stability Predicts Probe Subset-Sensitivity

• Lower SheshaFS predicts greater variability in probe accuracy across random feature halves, with raw ρ = −0.30 and partial ρ = −0.38.The relationship persists after controlling for mean accuracy and coefficient-of-variation normalization, linking stability directly to subset-based interpretability reliability.Source: §4.7 · pp. 17–18

5.1 Two Axes of Representational Geometry

• Geometric stability is a formally distinct axis because similarity metrics are rotation-invariant, while Shesha measures whether one representation’s geometry remains recoverable across feature subsets.Across 2,463 configurations in seven domains, the axes are governed by transformation regime rather than a single correlation; under compression, similarity remains high while stability falls.Source: §5.1 · p. 18, §5 · p. 18

5.2 Geometric Stability as a Distinct Selection Axis

• DINOv2 uniquely combines top-tier transfer with bottom-quartile stability, while transferability and stability show no general trade-off across 36 architectural families.The family-level association is Theil-Sen ρ = +0.21, not significant; DINOv2’s low stability reflects non-redundant variance distribution rather than concentrated eigenspectrum.Source: §5.2 · p. 21
• Stability should accompany transferability in model selection, especially when representations will be probed, steered, patched, or pruned, because high transfer does not ensure coordinate-level robustness.SheshaFS predicts how linear-probe accuracy depends on the feature subset, although stability does not predict input-distribution shift.Source: §5.2 · p. 21, §5.3 · p. 22

5.3 Relation to Mechanistic Interpretability

• Geometric stability exposes reliability assumptions in mechanistic interpretability: low SheshaFS warns that probe or steering targets may depend on the measured coordinates rather than the global representation.The diagnostic extends by mechanism to activation patching and steering, but those interventions were not directly tested; low-stability foundation models may therefore be unreliable test beds.Source: §5.3 · pp. 22–23

5.4 Geometric Stability Across Substrates

• The axis is substrate-independent: protein, molecular, and neural representations exhibit measurable geometric stability, with compression producing negative stability-similarity relationships and natural encoders showing negligible correlations.In neuroscience, stability complements noise-ceiling measures by diagnosing fragile geometry rather than unreliable data.Source: §5.4 · pp. 23–24

5.5 Limitations

• Shesha is global and may miss fragile submanifolds; benchmark rankings are mostly point estimates, transformation regimes are incomplete, biological coverage is uneven, and objective-level causal attribution remains associational.CIFAR-10 reproduced rankings across three seeds with Spearman ρ ≥0.993 and median per-model CV 0.75%, whereas the other five datasets used one seed.Source: §5.5 · pp. 24–25

5.6 Outlook

• The proposed outlook is to report S alongside accuracy, transferability, and robustness, while testing objectives or regularizers that increase coordinate-basis redundancy without sacrificing transfer.DINOv3 already raises stability at little or no transfer cost on four of six datasets, leaving residual transfer costs as an open question.Source: §5.2 · p. 21, §5.6 · p. 25

A.1 Feature-Split Shesha (SheshaFS)

• SheshaFS is the paper’s primary variant: it randomly splits feature dimensions into complementary halves, computes cosine-distance RDMs, and averages Spearman correlations over K=30 partitions.It measures redundant geometric encoding across the feature basis and requires neither labels nor repeated measurements; other variants are described but not used.Source: Appendix A · p. 25, §A.1 · p. 26

Method

• Sample-Split Shesha partitions data points into two disjoint subsets and computes RDM correlations using overlapping sample pairs or anchor-based approaches.This contrasts with feature splitting by testing consistency across observations rather than coordinates.Source: §A.2 · p. 26

Interpretation

• A low Sample-Split Shesha value may indicate sensitivity to sampling noise or reliance on spurious input-specific information.Source: §A.2 · p. 26

Scope

• Sample-Split Shesha is included for completeness but not used in the present paper, representing the observation-based complement to feature-split Shesha.Source: §A.2 · p. 26

Appendix B. Invariance Proofs and Counterexample

• The appendix formalizes these results for a representation matrix X, random feature partitions πk=(Ak,Bk), and cosine-distance RDMs computed on each half.These definitions provide the framework for proving Shesha’s invariance and non-invariance properties.Source: Appendix B · p. 26

B.1 Global Scaling Invariance

• Global positive scaling leaves every cosine-distance RDM unchanged and therefore preserves Shesha.For Y=αX with α>0, each partition and both feature halves produce the same cosine distances as X.Source: §B.1 · p. 26

B.2 Isotropic Scaling Invariance

• Isotropic scaling is likewise invariant because multiplying X by α does not change cosine distances.The proof follows directly from the global-scaling result.Source: §B.2 · p. 26

B.3 Feature Permutation Invariance

• Shesha is invariant to arbitrary feature permutations because uniformly random partitions are unchanged in distribution under coordinate relabeling.The corresponding split-half RDMs are reindexed, so averaging over independent partitions yields S(XP)=S(X).Source: §B.3 · pp. 26–27

B.4 Monotonic Distance Invariance

• Shesha is invariant to strictly increasing transformations of distances because Spearman correlation depends only on pairwise-distance orderings.Applying a monotone transformation leaves both rank vectors unchanged for every split.Source: §B.4 · p. 27

B.5 Non-Invariance to PCA Compression

• PCA compression reduces Shesha but can leave CKA near one when geometric information is concentrated in dominant components.When informative columns fall in one half of a random split, the other half-RDM is degenerate; such splits have positive probability for r ≤⌊d/2⌋.Source: §B.5 · p. 27

B.6 Non-Invariance to Orthogonal Transformations: Constructive Counterexample

• Shesha is not invariant to orthogonal transformations: an explicit rotation gives S(XQ) ≠ S(X) while CKA(X, XQ)=1.Any rotation that concentrates column energy into a strict coordinate subset can reduce Shesha without changing XX^⊤, with the reduction depending on axis-wise uniformity.Source: §B.6 · pp. 27–28

B.6.1 Numerical Confirmation at Fixed Spectrum

• S(X)=0.903 falls to S(Y)=−0.008 after rotating into the eigenbasis, while CKA remains 1.000 and the singular spectrum and rank are unchanged.The rotation concentrates variance in leading coordinates, leaving trailing near-zero coordinates that produce degenerate half-RDMs; the collapse therefore reflects the coordinate basis rather than the spectrum.Source: §B.6.1 · p. 28

Appendix C. Connection to RSA Noise Ceiling

• Shesha adapts RSA’s split-half RDM correlation across features rather than observations, measuring whether geometry is redundantly encoded from arbitrary feature subsets using a single representation.It uses K independent random equipartitions, unlike noise ceilings that require repeated measurements across trials or subjects.Source: Appendix C · p. 29

Appendix D. Shesha Computation

• The standardized protocol splits feature dimensions into equal random halves, computes cosine-distance RDMs, correlates their upper triangles with Spearman correlation, and averages K = 30 partitions.When necessary, samples were capped at nmax = 1,600, with fixed seed 320 and Float64 computations.Source: Appendix D · pp. 29–30

Appendix E. Ground Truth Validation

• SheshaFS recovers known stability with ρ = 0.997 and remains discriminant from debiased CKA, which falls to ρ = 0.204 in a balanced design.Geometry-preserving transformations move SheshaFS and CKA together, whereas geometry-altering transformations dissociate them.Source: Appendix E · p. 30, §E.3 · p. 32

E.1 Convergence Over K and Subsampling

• At n = 400, all 15 models produced stable Shesha estimates, with mean drift 0.0077 and overall mean absolute drift 0.0115 across 30 model-dataset combinations.Drifts ranged from 0.0002 for ResNet-50 to 0.0176 for ViT-Tiny; nmax = 1,600 was retained conservatively.Source: §E.1 · p. 31, Figure S1 · p. 31

E.2 Dimensionality Sensitivity

• Reducing embeddings to 64 PCA components changed mean SheshaFS from +0.620 to −0.112 across 30 conditions, supporting full-dimensional computation.The range after reduction was [−0.204, −0.055], because concentrated variance makes geometry unrecoverable from arbitrary coordinate halves.Source: §E.2 · p. 31

E.3 Sensitivity to Known Stability Levels

• Across 21 synthetic signal-to-noise levels, SheshaFS tracked parametrically controlled stability with near-perfect rank correlation, ρ = 0.997.The representations mixed a low-rank signal with isotropic noise, and α varied from 0 to 1 in increments of 0.05.Source: §E.3 · p. 32

E.4 Spectral Deletion

• After removing one dominant principal component, CKA, PWCKA, and Procrustes fell below 0.5, while SheshaFS retained meaningful signal through k = 20 and stayed positive at k = 50.This spectral deletion result shows that SheshaFS remains sensitive across the eigenspectrum rather than only to leading components.Source: §E.4 · p. 32, §E.5 · p. 33

E.5 Tail Noise Ablation / E.5.1 Protocol

• The tail-noise ablation perturbed only 478 non-signal components after the top 34 explained 90% of variance, testing CKA, RDM similarity, and internal SheshaFS across 14 noise scales.Representations had 2,000 samples and 512 features, with σ ∈ [0.001, 50] and three independent noise draws per condition.Source: §E.5.1 · p. 33

E.5.2 Results

• At σ ≤0.01, CKA and Shesha RDM similarity exceed 0.999, while internal SheshaFS remains at its baseline of 0.971.Source: §E.5.2 · p. 33
• At σ = 0.05, CKA and Shesha RDM similarity remain above 0.99, showing robustness to moderate tail-noise amplification.Source: §E.5.2 · p. 33
• As noise increases, CKA falls below 0.95 at σ = 0.20, while Shesha RDM similarity crosses that threshold slightly earlier at σ = 0.10.Source: §E.5.2 · p. 33
• At σ ≥1.0, both metrics collapse to near zero once injected tail energy overwhelms the original tail variance.Source: §E.5.2 · p. 33
• Across the full noise range, CKA and Shesha degrade in parallel, with no regime where Shesha detects a change that CKA misses.This indicates Shesha’s sensitivity to spectral tail structure is specific to genuine removal of signal-carrying components, not energetic perturbations of non-functional dimensions.Source: §E.5.2 · p. 33

E.5.3 Interpretation

• Spearman rank-order correlation explains why SheshaFS ignores rank-preserving noise but degrades when noise scrambles pairwise-distance rankings.Monotone distance transformations leave S unchanged (Table 1, monotonic distance invariance); once rankings are scrambled, CKA degrades equally.Source: §E.5.3 · p. 33

E.6 Preprocessing Ablation

• The preprocessing study evaluates raw, centered, L2-normalized, and whitened representations.Whitening uses ZCA with shrinkage λ = 0.1.Source: §E.6 · p. 34
• The divergence is therefore robust across three non-whitened preprocessing conditions.These conditions are raw, centered, and centered with L2 normalization.Source: §E.6 · p. 34
• The Shesha–CKA divergence persists under raw, centered, and centered-with-L2-normalization preprocessing, but not whitening.Whitening equalizes the spectrum, eliminating the divergence.Source: Table S1 · p. 34, §E.6 · p. 34

E.6.1 Mechanistic Interpretation of Whitening

• Whitening equalizes the spectrum and uniquely disrupts the Shesha–CKA divergence.Source: Table S1 · p. 34
• Under whitening, CKA remains negative at k = 30 (−0.054), compared with −0.076 under raw preprocessing.Source: §E.6.1 · p. 34
• Whitening reduces the Shesha baseline from 0.98 to 0.50 at k = 0.The drop reflects noise amplification caused by spectral equalization.Source: §E.6.1 · p. 34

E.7 Comparison with RSA Reliability Methods; B. Shesha vs Whitened Variant; C. Shesha Across Preprocessing

• ρ = 1.000 between standard and whitened Shesha, confirming consistency with established RSA reliability practices while avoiding whitening-induced numerical instability and noise amplification.Standard Shesha operates directly on raw representations without requiring ZCA whitening.Source: §E.7 · p. 34, B. Shesha vs Whitened Variant · p. 36, C. Shesha Across Preprocessing · p. 36

E.7 Comparison with RSA Reliability Methods; A. Stability vs Similarity Metrics; C. Shesha Across Preprocessing; B. Shesha vs Whitened Variant; D. CKA Across Preprocessing

• An orthogonal rotation that concentrates variance into selected coordinates lowers SheshaFS while leaving CKA unchanged, showing that the divergence reflects basis dependence rather than spectrum alone.The result supports Shesha’s sensitivity to geometric structure distributed across the eigenspectrum, while similarity metrics are dominated by top principal components.Source: §E.7 · p. 34, Table S2 · p. 35, Figure S3 · p. 36

E.8 Seed Stability

• Mean seed sensitivity was 0.0047, with a maximum of 0.0142, and all 30 model–dataset combinations remained below the 0.05 stability threshold.Twenty-five of 30 combinations were below 0.01 when K = 30 random feature partitions were averaged across two seeds.Source: §E.8 · pp. 36–37

E.9 Dissociation with Balanced Quadrant Sampling

• ρ = 0.204 between Shesha and debiased CKA under balanced four-quadrant sampling, demonstrating that stability and similarity assess largely different attributes.The four quadrants separately realize high/high, high/low, low/low, and low/high stability–similarity combinations; the adversarial low-stability/high-similarity quadrant gives Shesha = −0.001 ± 0.005 and CKA = 0.978 ± 0.000.Source: §E.9 · pp. 37–38, Figure S5 · p. 38

F.1 Cross-Domain Validation: Data Sources and Preprocessing; F.1.1 Language (N=127); F.1.2 Vision (N=129); F.1.3 Audio (N=64); F.1.4 Video (N=128); F.1.6 Molecular (N=767); F.1.7 Neural Population Recordings (N=846)

• The corpus spans seven domains, from natural language to neural population recordings, using fixed stimulus sets and domain-specific configurations that vary encoders or encoding schemes and preprocessing.Source: §F.1 · p. 39

F.1.4 Video (N=128)

• The video analysis reproduced the stability–similarity relationship across sources, with ρ = −0.24 for a single-source Jellyfish sample versus ρ = −0.27 for the main video corpus.This comparison used the same base models and preprocessing on 100 uniformly sampled clips.Source: §F.1.4 · p. 39

F.2 Encoder Transformations; Appendix F. Encoders; F.1 Cross-Domain Validation: Data Sources and Preprocessing

• Across 2,463 configurations in seven domains, distance-preserving transforms keep CKA and Shesha high, whereas PCA compression preserves CKA but collapses Shesha, producing negative coupling.The pooled correlation is near zero because only the variance-concentration regime produces negative coupling.Source: Appendix F · p. 38, §F.1 · p. 39, §F.2 · p. 40

F.3 Similarity Metrics; F.3.1 Effective-Rank Projection-Weighted CKA (PWCKA); F.3.2 Procrustes Similarity

• Alternative similarity metrics, including PWCKA and Procrustes similarity, maintained |ρ| < 0.30 with Shesha in the language domain, generalizing their distinctness beyond CKA.PWCKA aligns representations after projection to an effective rank explaining 99% of variance, while Procrustes optimally aligns normalized representations orthogonally.Source: §F.3.1 · p. 41, Table S3 · p. 42, §F.3.2 · p. 42

F.4.1 Bootstrap Inference

• Distinctness used Spearman correlations with 10,000 bootstrap replicates within domains and 95% bootstrap percentile confidence intervals.The resampling scheme was applied to encoder configurations within each domain.Source: §F.4.1 · p. 42

F.4.2 Mixed-Effects Models

• A mixed-effects model found base-model identity explained 10% of stability variance, while CKA’s fixed-effect slope was −0.03 (95% CI [−0.08, +0.02]).Stability S was the outcome, debiased CKA the fixed effect, and base model a random intercept.Source: §F.4.2 · p. 42

F.4.3 Mann-Whitney U Tests

• Architectural comparisons used two-sided Mann-Whitney U tests on SheshaFS scores and reported exact p-values.The comparisons covered contrastive versus self-supervised and hierarchical versus columnar architectures.Source: §F.4.3 · p. 43

F.4.4 Multiple Comparisons

• Vision-benchmark tests were reported without multiplicity correction because each dataset was treated as an independent evaluation domain.This procedure avoided treating datasets as repeated tests of the same hypothesis.Source: §F.4.4 · p. 43
• ρ = −0.01 (95% CI [−0.06, +0.03]) across 2,463 encoder configurations and seven domains, confirming negligible net correlation between Shesha and CKA.Robustness checks maintain |ρ| < 0.10, while aProtein’s moderate negative correlation is attributed to PCA on low-dimensional sequence encoders.Source: Figure S7 · p. 43, Table S5 · p. 44, Table S6 · p. 44

Appendix G. Vision Benchmark: Extended Results

• The vision benchmark specified how 170 models were selected and how their features and transferability scores were computed, with subsections documenting these choices and extended analyses supporting the main-text claims.Source: Appendix G · p. 44

G.1 Model Selection

• The 170 models spanned objectives, architectural families, scales, and training paradigms, and were grouped into 36 semantic families with training objective prioritized when classifications conflicted.The selection included supervised, self-supervised, contrastive, generative, convolutional, hybrid, columnar, and hierarchical models.Source: §G.1 · p. 44

G.2 Feature Extraction

• Penultimate-layer features were extracted from fixed random dataset subsets using seed 320 and each model’s standard resize, crop, and normalization transform; LogME and, where applicable, LEEP measured transferability.LogME used the authors’ implementation, while LEEP was computed only for models with classification heads.Source: §G.2 · p. 45, §G.3 · p. 45

• DINOv2-giant ranks bottom-quartile in SheshaFS on every dataset except EuroSAT, despite top-six transferability on CIFAR-10, CIFAR-100, and Flowers-102.On EuroSAT, it is among the most stable models, ranking 4/170.Source: Table S7 · p. 45
• Contrastive models are more stable than self-supervised models on all six datasets.The comparison includes CLIP, SigLIP, and ViTamin contrastive models (n=29) versus self-supervised models (n=41), using Mann-Whitney U tests on SheshaFS.Source: §G.4 · p. 45, Table S8 · p. 45
• Family-level transferability and stability trends are positive or null on five datasets and negative only on Flowers-102, so no consistent trade-off emerges.Flowers-102 shows the sole trade-off and the sharpest DINOv2 dissociation; EuroSAT is the exception where DINOv2 ranks among the most stable families.Source: Figure S8 · p. 46

G.5 Seed Stability of Vision Models

• SheshaFS rankings are nearly seed-invariant across 170 models, with Kendall τ=0.944–0.945 and Spearman ρ=0.993–0.995 between seeds.The median per-model coefficient of variation is 0.75%, and only 5 of 170 models exceed 5% CV; all five have absolute SheshaFS standard deviation below 0.06.Source: §G.5 · p. 47, Table S10 · p. 48
• The DINOv2 paradox is seed-invariant: the register-augmented large variant ranks 170th under all three seeds, while DINOv2 has the lowest family-mean SheshaFS each time.Its SheshaFS values are 0.291, 0.287, and 0.297 across seeds 9, 320, and 1991.Source: §G.5 · p. 47
• Despite a significant Friedman test for SheshaFS seed effects (χ2=8.48, p=0.014), sub-percent shifts leave rankings essentially unchanged, so seed 320 is used in the main text.The negligible magnitude is reflected by the 0.75% median CV and near-perfect rank agreement.Source: §G.5 · p. 47, §G.6.1 · p. 48

G.6 SAM vs. SGD Ablation: Full Protocol and Extended Results

• The SAM ablation tests whether SheshaFS responds to optimization geometry by changing the flatness penalty while accuracy and learned features remain approximately constant.SAM maximizes loss within an ℓ2-ball of radius ρ before each gradient step, whereas ρ=0 recovers standard SGD.Source: §G.6 · p. 47, §G.6.1 · p. 48

G.6.1 Protocol

• The protocol trains CIFAR-10 and CIFAR-100 ResNet-18 models with identical optimization settings, varying only SAM’s perturbation radius across six values from 0 to 0.2.Training uses learning rate 0.05, momentum 0.9, weight decay 5 × 10−4, batch size 128, cosine annealing, and 100 epochs.Source: §G.6.1 · pp. 47–48
• Each SAM configuration is trained over 15 random seeds, with 512-dimensional penultimate representations evaluated on 2,000 test images using accuracy, debiased linear CKA, SheshaFS, and supervised Shesha variants.The supervised contrasts are variance ratio, supervised alignment, and class-separation ratio; all metrics are reported as mean ± SD over the 15 seeds.Source: §G.6.1 · p. 48

G.6.2 Results

• +0.067 and +0.018: at peak perturbation radii, SheshaFS exceeds the SGD baseline on every seed while CKA falls from 1.000 to 0.925 on CIFAR-10 and 1.000 to 0.772 on CIFAR-100.The CIFAR-10 result occurs at ρ = 0.05 and the CIFAR-100 result at ρ = 0.2; the full results are reported over 15 seeds in Tables S11 and S12.Source: §G.6.2 · p. 48, Table S11 · p. 49, Table S12 · p. 49
• The intervention therefore holds accuracy fixed while moving CKA and SheshaFS in opposite directions, and SAM raises split-half consistency by distributing features more uniformly across coordinates.This dissociation distinguishes coordinate-basis redundancy from variance concentration along class-discriminative directions.Source: §G.6.2 · pp. 48–49
• SheshaFS has an interior, dataset-dependent optimum: 0.872 at ρ = 0.05–0.1 on CIFAR-10 versus 0.822 at ρ = 0.2 on CIFAR-100.On CIFAR-10 it declines to 0.851 at ρ = 0.2, while the variance ratio reverses from 0.786 at ρ = 0.1 to 0.790 at ρ = 0.2, indicating coordinate-variance redistribution rather than seed noise.Source: §G.6.2 · p. 48
• Label-aware metrics move opposite to SheshaFS: on CIFAR-10, variance ratio falls from 0.849 to 0.790 and class-separation ratio from 3.08 to 2.41 while supervised alignment stays near 0.51.CIFAR-100 shows the same pattern, supporting SheshaFS as a measure of coordinate-basis redundancy rather than classification geometry.Source: §G.6.2 · p. 49

G.7 Probe Subset-Sensitivity: Full Protocol and Stratified Analysis

• The diagnostic protocol evaluates 170 vision models using fixed stratified CIFAR-10 samples, 20 random half-feature probes per model, and SheshaFS computed with K = 30 splits.Representations contain 512 to 1536 penultimate-layer dimensions; standardization uses training statistics, and the sample split remains fixed across probes.Source: §G.7 · p. 49, §G.7.1 · p. 49

G.7.2 Headline Result

• ρ = −0.302 and −0.260: across 170 models, SheshaFS predicts probe-accuracy standard deviation and range, with controls ruling out a ceiling artifact.The coefficient-of-variation correlation is ρ = −0.280, while controlling for mean probe accuracy strengthens the association to ρpartial = −0.382.Source: §G.7.2 · p. 50

G.7.3 Stratified analysis

• ρ = −0.473: the SheshaFS–probe-variability relationship is strongest in the middle accuracy tercile and attenuates at low and high accuracy.The correlations are ρ = −0.261 for high accuracy and ρ = −0.072 for low accuracy; DINOv2 models at SheshaFS ≈0.29 to 0.37 produce uniformly mediocre probes because geometry is distributed non-redundantly across coordinates.Source: §G.7.3 · p. 50