PaperlayerView exact revision on arXiv

Source-linked AI summary

Certified Topological Interaction in Neural Representations: Exact Tests and the Statistic They Require

Sushovan Majhi

arXiv:2609.08561v2cs.LGmath.AT

TL;DR

Comparative claims about class disentanglement are often based on descriptive curves without a null distribution. The paper adds exact tests and a guarded certificate to a topological measure of class-cloud overlap, and reports that joint triple entanglement is often below the strongest pairwise entanglement.

  • Problem

    Representation analyses often lack a null distribution for claims that classes become more disentangled across layers or epochs, and existing measures do not capture joint overlap among multiple classes.

    Source: §1 · p. 1, §1.2 · p. 3

  • Method

    The paper measures scale-dependent class-cloud overlap with the Intersection ECP and adds exact permutation tests, a guarded separation certificate, and a paired test on a scale-free statistic.

    Source: §3.2 · p. 9, §5 · p. 23

  • Results

    Joint triple entanglement is below the strongest pair in 97% of triple–layer cells and 99.5% of deep cells, a regularity reported in vision encoders and frozen language models.

    Source: §5 · p. 23

  • Takeaways & Limitations

    The triple-versus-pair pattern is a regularity rather than a law, and the unnormalized profile mass predicts test accuracy while the quotient does not.

    Source: §5 · p. 23

    Measurements use PCA shadows of dimension d0 ≤6; whether class supports are well approximated in the projection is unverified, and overlap in the shadow may be an artifact.

    Source: §5 · p. 23

Abstract

from arXiv · show

Class disentanglement--the separation of a representation's class-conditional point clouds along depth and over training--is measured by descriptive curves: the sentence such a study wants to write, layer l+1 is more disentangled than layer l, is an eyeball judgement with no null. We supply the inferential layer for a topological measurement of class overlap, the Intersection Euler Characteristic Profile: the Euler characteristic of the overlap of the clouds' ball unions as a function of scale, from one Alpha-complex sweep with no boundary-matrix reduction. Every number carries a test--exact permutation tests in both directions, a guarded separation certificate the invariant requires, and a paired sign-flip test for comparative claims. Building that test taught a lesson outliving this invariant: its statistic must be scale-free. On the raw profile mass, which has units of feature length, 12,375 paired tests return 5,633 significant steps of which every one at the first epoch points the wrong way, certifying feature-norm dynamics as disentanglement; the dimensionless statistic returns 2,707, with 2,026 decreases. Across 111 networks and 52,650 measurements, disentanglement is depth-graded and early, and interaction quotients rank class pairs by confusability (rho=0.83), on par with cheap separability statistics. In a 96-model factorial, augmentation is the one training choice that separates classes relative to chance; weight decay compresses the overlap without separating. Only a k-fold statistic can pose the structural question: the joint entanglement of a class triple sits below its strongest pair in 97% of triple-layer cells and 99.5% of deep cells, at median ratios far below a measured null floor, in vision encoders and frozen language models--a regularity, not a law. The unnormalized mass predicts test accuracy (R^2=0.94), the quotient does not, and neither beats a linear probe.

1 Introduction

The paper treats disentanglement as the separation of class-conditional feature clouds and adds statistical tests to topological measurements of their overlap. It reports depth-graded and early separation, a scale-free comparative statistic, and evidence that joint triple entanglement is often weaker than pairwise entanglement.

1 Introduction

  • A classifier maps images to feature-space points, and the paper asks how much class clouds overlap at each layer and scale, and whether observed changes exceed chance.Source: §1 · p. 1

1.1 Class separation, and how it has been measured

  • Existing measures describe class separation or representation geometry, but do not measure class-cloud overlap; several lack null distributions or a joint statistic for more than two classes.The mixup barcode is pairwise, asymmetric, descriptive, and costly; the paper identifies comparative inference for representation analyses as the gap addressed here.Source: §1.1 · p. 2, §1.2 · p. 3

1.3 Our contributions

  • The Intersection ECP measures class-cloud overlap symmetrically, for any number of classes, using one Alpha-complex sweep without matrix reduction and exact permutation tests.The protocol adds guarded separation certificates and paired sign-flip tests for comparative claims.Source: §1.3 · p. 4
  • Raw profile mass produced 5,633 significant epoch-to-epoch steps and pointed the wrong way at every significant first-epoch step; the dimensionless statistic certified 2,707 steps and pointed the right way.The permutation-null mean of the mass grew twentyfold as features spread, motivating a scale-free statistic for paired comparisons.Source: §1.3 · p. 4
  • Disentanglement is depth-graded and early: deep layers do most separating, and most certified changes occur within the first eight epochs.The interaction quotient ranks MNIST pairs by confusability at Spearman ρ=0.83; a from-scratch ViT is non-monotonic, an artifact removed by pretraining.Source: §1.3 · p. 4
  • Joint triple entanglement is below the strongest pair in 97% of triple–layer cells and 99.5% of deep cells, but this is a regularity, not a law.The median ratios are far below the measured null floor; the pattern is reported in vision encoders and frozen language models.Source: §1.3 · p. 4
  • In a 96-model factorial, augmentation separates classes relative to chance, while weight decay compresses overlap without separating; the unnormalized mass predicts test accuracy (R^2 = 0.94).The quotient does not predict test accuracy, and neither statistic beats a linear probe.Source: §1.3 · p. 4
  • Two implementations of a differentiable surrogate steer the wrong way, supporting the authors’ choice to keep the certificate non-differentiable.Source: §1.3 · p. 4

2 The Intersection ECP

The Intersection ECP is a scale-indexed Euler characteristic of the overlap among thickened class clouds. Its strengths include symmetry, applicability to multiple classes, and a cheap computation; however, Euler-characteristic cancellation can make real overlap invisible.

2.1 The invariant and its properties

  • The Intersection ECP is the Euler characteristic of the region where thickened class clouds overlap, evaluated as a function of scale.For two clouds it is the inclusion–exclusion defect of their Euler characteristic curves.Source: §2.1 · p. 5
  • The paper supplies an exact, distribution-free permutation test as the inferential layer for an invariant whose stability and sampling theory were established previously.The test can assess whether clouds interact more or less than chance for any sample size.Source: §2.1 · p. 6
  • The profile is symmetric and defined for any number of classes, and a single sorted Alpha-complex sweep computes it without boundary-matrix reduction.The sweep uses signed simplex counts.Source: §2.1 · p. 6

2.2 What Δχ cannot see

  • Balanced Betti numbers can make an annular or toroidal overlap have χ=0 across a scale plateau, indistinguishable from no overlap.The first interaction scale distinguishes an empty intersection from cancellation; when the profile is silent despite nonempty overlap, the paper points to overlap Betti curves or relative homology.Source: §2.2 · p. 6

3 Certified measurement of disentanglement

The section turns the Intersection ECP into a tested measurement of class-cloud disentanglement, with safeguards for cancellation and feature-scale changes. It extends the measurement to comparative claims and multi-class structure while controlling estimation and multiplicity.

3 Certified measurement of disentanglement

  • The section specifies class clouds, projection, grid, and interaction quotient as the measurement protocol, with every reported number tied to a test.PCA projection can make classes appear more entangled but not more separated; permutation certificates concern the projected clouds.Source: §3 · p. 6, §3.1 · pp. 7–8

3.1 Class clouds, projection, and the interaction quotient

  • The quotient normalizes profile mass by its exchangeable-null mean, making it dimensionless, centered at 1 under exchangeability, and tending to 0 under separation.The paired tests instead use dimensionless profile mass divided by the grid maximum; raw mass has feature-length units.Source: §3.1 · p. 8, §3.3 · p. 9

3.2 Two one-sided tests, and the guarded certificate

  • Exact one-sided permutation tests distinguish excess interaction from separation, but a separation certificate is issued only when the guard is silent.The guard withholds certification when the profile is silent just beyond first contact, where sampled offsets provably meet; Figure 3 illustrates annular overlap with zero Euler characteristic.Source: Figure 3 · p. 7, §3.2 · pp. 8–9

3.3 Comparative claims: the paired test

  • Comparative claims use paired folds and an exact sign-flip test; with 8 folds, the smallest attainable two-sided p-value is 2/256 ≈ 0.008.The sign of the mean paired difference is interpreted after the two-sided test.Source: §3.3 · p. 9

3.4 Estimation and multiplicity

  • The sample profile equals the population profile at regular scales once samples are sufficiently dense, with sample complexity depending on intrinsic support dimension.Benjamini–Hochberg controls false discovery rate at q=0.05 within claim families under an assumed positive-regression-dependence condition; the authors do not prove that condition.Source: §3.4 · p. 10

3.5 Multi-class structure

  • The multi-class interaction spectrum extends pairwise measurement to joint groups; triple and higher-order terms detect classes jointly entangled when no pair is.Its graded decay quantifies pairwise dominance, and onset delays record how much later multi-class overlap begins than pairwise contact.Source: §3.5 · p. 10

4 Experiments

The experiments evaluate scale-free topological statistics for testing disentanglement and comparing pairwise with higher-order class entanglement, alongside their calibration, guard, confusability-ranking, generalization-prediction, and surrogate properties. They also examine how training choices, projection dimension, and frozen language models relate to class overlap and pairwise dominance.

E1 — reproduce Wagner, at scale

  • For 45 MNIST digit pairs, the interaction quotient’s confusability ranking correlated with classifier confusion at Spearman ρ=0.83 (p=2.4×10^-12), comparably to cheap statistics.All 45 pairs were certified separated; 5-NN cross-class rate had ρ=0.86, probe error 0.83, energy distance 0.77, and Fisher ratio 0.74. Ranking stability across the protocol sweep had median ρ=0.87 and minimum 0.73 over 80 cells.Source: §4 · pp. 11–12, Figure 4 · p. 12

E2 — multi-class disentanglement vs depth

  • Adding the triple quotient changed leave-one-out R^2 by a median of −0.004 across 18 cells and improved it in only 4, by at most +0.04.The pairwise-quotient regression reached R^2=0.71 in the CIFAR-10 ResNet-56 deep stage and 0.76 at the ViT penultimate layer; the triple quotient had partial rank correlation p<0.05 in 7/18 cells but did not yield out-of-sample prediction gain.Source: §4 · pp. 13–14
  • The ResNet-56 CIFAR-10 mean pairwise quotient fell from 0.72 at the stem to 0.18 at stage 3, while the from-scratch ViT profile was non-monotone.The CIFAR-100 subset fell from 0.79 to 0.06 at deep layers. The ViT quotient reached 0.31 at block 6 then rose to 0.39 at the penultimate representation; a pretrained ViT-B/16 finetune instead had a monotone profile, already present at epoch 0.Source: §4 · pp. 12–13, Figure 5 · p. 15
  • In the 1,560 distinct triple-layer cells, the joint quotient was no greater than the largest pairwise quotient in 96.7%, with a median triple-to-maximum-pair ratio of 0.63.The deep rescan found dominance in 597/600 cells (99.5%; median ratio 0.47), and no cells met the specified strong counterexample criteria. Against the measured null floor, cell-median tests gave p=3.3 × 10^-8 for the full scan and p=1.9 × 10^-5 for the deep rescan. This is a representation-specific regularity, not a geometric theorem: an adversarial Gaussian search found a ratio of 1.98 at its optimum and a median of 1.24 over 20 draws.Source: §4 · pp. 13–14, Figure 6 · p. 16

E3 — training-dynamics trajectories at scale

  • 2,707 of 12,375 paired steps were BH-significant on the scale-free statistic, with 2,026 decreases; training disentangles mainly early and in deep stages.There were 687 significant steps at epoch 0→1 and 1,728 within the first eight epochs; the deep stage had 873 certified decreases versus 42 at the stem. The raw profile mass instead produced 5,633 significant steps, including 1,125 increases at epoch 0→1, all opposite the quotient’s direction.Source: §4 · pp. 14–15, Figure 7 · p. 17

E4 — calibration, and the guard

  • All three tests were calibrated within binomial error of nominal rejection rates, and the cancellation guard fired zero times across 52,650 pairwise measurements.The tests were the two one-sided tests and the paired sign-flip test; the zero guard firings mean the monitored false-certificate pattern was not observed, not that Euler cancellation is impossible elsewhere in the scale range.Source: §4 · pp. 16–17, Figure 8 · p. 18

E6 — steering disentanglement with a differentiable surrogate

  • The differentiable surrogate raised certified entanglement as its weight increased: 0.159→0.344 when detached and 0.167→0.488 when attached, so it did not yet provide a usable regularizer.The detached surrogate was gamed by feature-norm inflation, while the attached version was gamed by within-class dispersion; the certified quotient exposed both failures.Source: §4 · pp. 18–19

E5 — predicting generalization

  • On 96 models, unnormalized ECP features predicted test accuracy at R^2=0.94, whereas null-normalized quotient features reached R^2=0.34.Total mixup reached R^2=0.84; adding either ECP feature family or mixup to the baseline left R^2 at 0.985–0.988 versus 0.987. Neither topological feature family added information beyond the cheap probes.Source: §4 · pp. 17–18, Table 1 · p. 19

E7 — the interaction spectrum

  • The interaction spectrum decayed monotonically with class-order in every layer, with mean step ratios of approximately 0.88 at the stem and 0.61 at the deepest stage.Ten-fold overlap began at twenty times the pairwise scale in the deep stage; top-floor quotients were 0.015–0.07 in deepest stages versus 0.44–0.73 at the stem.Source: §4 · p. 19

E8 — robustness to the projection dimension

  • At projection dimensions 3, 4, and 6, every qualitative depth conclusion persisted, and pair-ranking median Spearman correlations against dimension 5 were 0.91, 0.96, and 0.97.Deep layers remained fully certified separated; lower correlations of 0.56–0.69 occurred only at dimension 3 among pairs already indistinguishable from zero.Source: §4 · p. 19

E9 — is pairwise dominance present at initialization?

  • At initialization, every triple was dominated in 120/120 cases for ResNet-20, ResNet-56, and ViT, with median ratios of 0.76, 0.76, and 0.78.Training deepened dominance to a median ratio of 0.47 in E2 deep layers; CIFAR pixels and a random network had similar pairwise quotients, 0.742 and 0.744.Source: §4 · p. 20
  • The measured null floor was a median ratio of 0.93 for Gaussian clouds and 0.97 for initialized features with permuted labels; memorization produced a 0.34 median ratio in the last stage.All initialization and trained deep cells sat below the null floor; the random-label network’s earlier stages remained exchangeable, while its last stage certified all 45 pairs as separated.Source: §4 · p. 20

E10 — outside vision: two frozen language models

  • BERT’s final-layer median ratio was 0.46 against 0.47 for trained deep-vision E2; across six language-model cells, every median was below every null-seed median.On DBpedia, BERT certified all pairs at both depths and dominated every triple; GPT-2 certified 44 of 45 pairs and dominated every triple.Source: §4 · p. 21, Table 2 · p. 21

E11 — what changes certified disentanglement

  • Augmentation lowered the deep-stage interaction quotient for 44 of 45 class pairs (mean 0.230 → 0.171), while overlap mass fell in 1,307 of 2,160 paired tests.It also lowered the quotient for 24 pairs at stage 2, 11 at stage 1, and none at the stem; its mean accuracy gain was +0.055.Source: §4 · p. 22, Table 3 · p. 22
  • Weight decay reduced stage-3 overlap mass in 1,743 of 2,160 tests, yet the interaction quotient rose for 32 of 45 pairs (+0.02).Weight decay increased accuracy by 0.033, separating accuracy gains from relative topological separation.Source: §4 · p. 22, Table 3 · p. 22
  • Increasing depth from 20 to 56 raised the quotient for 25 pairs at stage 2 and 11 at stage 3 by 0.02–0.04, with no accuracy change.Doubling width moved the quotient of at most 6 pairs at any stage, despite shrinking overlap mass at stages 1 and 3.Source: §4 · p. 22, Table 3 · p. 22

5 Conclusion and limitations

The campaign finds certified, depth-graded and early class disentanglement, while triple interactions show a recurring structure across vision and frozen language models. These conclusions are bounded by projection, reach, sampling, and measurement limitations.

Conclusion

  • At the penultimate stage, mean quotient falls 0.78 →0.16, versus 0.76 →0.73 at the stem; disentanglement is mostly significant within the first eight epochs.Input-side interaction quotients correlate with confusion at 𝜌= 0.83, on par with a nearest-neighbour rate.Source: §5 · p. 23
  • On raw profile mass, 5,633 paired steps were significant, with every first-epoch step pointing the wrong way as feature norms shrank under weight decay.The paper identifies scale-free statistics as necessary for interpreting paired trajectory tests.Source: §5 · p. 23
  • Joint triple entanglement is below its strongest pair in 97% of triple–layer cells and 99.5% of deep cells, at median ratios far below the null floor.The structure appears at initialization, deepens with true-label training, and is reproduced on two frozen language models; it is a regularity, not a law.Source: §5 · p. 23
  • Among 96 models, augmentation separates classes relative to chance, while weight decay compresses overlap without separating classes.Depth and width do nothing in this population.Source: §5 · p. 23

Limitations

  • Measurements use PCA shadows of dimension d0 ≤6; the projection-support approximation is unverified, and an ambient triple could be missed by the scan.The certificates and paired comparisons are exact for the sample, but the population-profile guarantee assumes positive reach and an 𝜖-dense sample, neither established for these projected clouds.Source: §5 · pp. 23–24
  • In wide, high-effective-rank representations, the scale window narrows with depth, and projected certificates retain only 8–26% of ambient interaction scale.The paper limits its claims to small trained vision classifiers, their initializations, and two frozen language models; it recommends checking the window before interpreting measurements on wider representations.Source: §5 · pp. 23–24
  • For pairwise ranking or separation timing, nearest-neighbour cross-class rates perform as well as the quotient; the quotient’s departures do not predict accuracy, calibration, or robustness.The topological statistic’s case rests on its null-based certificate, k-fold terms, scale resolution, and a from-scratch ViT head where cheap statistics disagree.Source: §5 · p. 24
  • The confusability ranking stays stable across tested sample sizes and dimensions, but the 200-point grid resolution was not varied.The guard and plateau form did not fire on any measured pair, yet structurally cancelling overlaps remain invisible to Δ𝜒 and the guard is not optional.Source: §5 · p. 24

A The inference used, with sources

This appendix fixes notation for the guarantees used in Section 3, defining the profile statistic, overlap region, and first-contact scale.

A The inference used, with sources

A.1 Exact tests in both directions

The appendix establishes exact one-sided permutation tests under exchangeable labels and gives the paired sign-flip test used for comparisons across conditions.

A.1 Exact tests in both directions

  • Under exchangeable pooled labels, both one-sided permutation p-values control type-I error at level α for any statistic, sample sizes, and number of permutations.The result also holds for k clouds split into their respective sample sizes.Source: §A.1 · p. 25
  • Conditioned on the pooled data, observed and permuted statistics are exchangeable, making the Monte Carlo rank p-value valid at every permutation count.The PCA projection and scale grid depend only on the pooled data, so the result applies to projected clouds on that grid.Source: §A.1 · p. 25

A.2 The paired test

  • For paired conditions evaluated on shared-input folds, the comparative null is invariance of the fold differences under coordinatewise sign changes.This null holds when conditions are exchangeable within each fold given shared preprocessing; independence across folds is not required.Source: §A.2 · p. 25
  • The paired sign-flip test controls its p-value for every fold count; with F=8, the p-value floor is 2/256.The sign of the mean difference is read after the two-sided test.Source: §A.2 · p. 25

A.3 Estimation

The estimation guarantee connects the sampled intersection profile to a population profile under reach, sampling-density, and filtration assumptions, while leaving one uniform-control issue open.

A.3 Estimation

  • For compact supports with positive reach and finite positive Hausdorff measure, the proposition assumes samples from densities bounded below and a population overlap filtration with finitely many critical values and finite-dimensional homology.Source: §A.3 · p. 25
  • Given 𝜖-dense samples, the sample profile equals the population profile across a band below the reach, except near population critical values and sample-artifact scales.The stated sample-size rate is in the intrinsic dimension.Source: §A.3 · p. 25
  • Uniform control of the sample-artifact set’s size as sample size grows remains open in the foundations paper and is carried as a hypothesis in Theorem 1.Source: §A.3 · p. 25

A.4 The blind spot and the guard

An Euler-characteristic profile can be silent both when class clouds are separated and when their overlap has cancelling Betti numbers, so an unguarded separation test can falsely certify separation. The guard uses the estimated first-contact scale to withhold claims when the profile is silent just beyond contact.

The blind spot

  • A χ-blind pair has a nonempty overlap with Euler characteristic zero across an interval, making its profile indistinguishable from that of a separated pair.Annular and torus-shell overlaps can be blind across whole intervals of regular scales.Source: §A.4 · p. 26
  • On a χ-blind configuration, the unguarded separation test rejects with probability tending to one even though the class clouds overlap at every scale from first contact onward.The result holds under the theorem’s stated hypotheses.Source: §A.4 · p. 26
  • The permutation null can produce non-cancelling overlap even when the labeled profile is zero, and its required null-statistic condition must be checked for each configuration.Sufficient cases include a pooled support with nonzero Euler characteristic on a positive-measure subset of the grid, or a finite-sample regime where random splits meet before labeled clouds do.Source: §A.4 · p. 26

The guard

  • The guard uses the first-contact estimate, which lies between population first contact and that scale plus ε for ε-dense samples, to check profile silence just beyond contact.The guard withholds a separation claim when the profile is silent on the specified window.Source: §A.4 · p. 26
  • The campaign guard uses the quarter-grid window immediately beyond estimated contact, but the stated mechanism gives no broader finite-sample guarantee.The appendix reports that neither guard form fired in the campaign’s cells.Source: §A.4 · p. 27

A.5 The concentration measurements

Table 4 describes concentration and certificate measurements on frozen representations, including ambient dimension, effective rank, scale window, and projected-certificate tightness. Its scale-window comparisons are qualified by sample-cap dependence and model-specific interpretability.

Measurements

  • For vision transformers, the scale window is reported separately for class tokens and mean-pooled patch tokens from the same forward pass.The window is a fractional gap between the first-percentile cross-class distance and the closest cross pair, using 100 points per class.Source: §A.5 · p. 27

Caveats

  • Scale-window magnitudes depend on the sample cap, while within-model layer ratios are stable across caps; GPT-2’s final-layer window is uninterpretable because its effective rank is near 1.The ResNet-50 stage-1-to-stage-4 ratio is 3.4× at 100 and 3.6× at 200 points per class.Source: §A.5 · p. 27

Measurements

  • Table 4 reports ambient dimension, participation-ratio effective rank, scale window, and projected-certificate tightness for frozen representations.The representations come from CIFAR-10 test images through vision models and ten classes of 20 Newsgroups through language models.Source: §A.5 · p. 27, Table 4 · p. 28

B Experiment E6 in full: the differentiable surrogate

Experiment E6 tests whether a differentiable surrogate can control disentanglement. Both tested implementations decrease the surrogate while increasing certified entanglement, exposing scale and dispersion as ways to game the loss.

Detached bandwidth

  • With detached σ², the certified quotient rose 0.159 → 0.212 → 0.278 → 0.344 as λ increased, while test accuracy fell 92.0% → 86.5%.A fixed bandwidth makes the radial gradient negative, inflating feature norms despite the loss value being scale-invariant.Source: B Experiment E6 in full: the… · pp. 28–29

Corrected campaign

  • Keeping σ² in the graph closes the radial scaling direction, but the loss can still fall as within-class radii grow and the class clouds interpenetrate.The exact ECP measures the resulting increase in entanglement.Source: B Experiment E6 in full: the… · p. 29
  • With σ² in the graph, the quotient rose 0.167 → 0.250 → 0.488 at λ=0, 0.5, 2, while test accuracy fell 92.1% → 91.0% → 65.7%.At λ=8, test accuracy averaged 16%; one seed collapsed to a single feature vector and the other had quotient 0.88.Source: B Experiment E6 in full: the… · p. 29

Results

  • Both surrogate implementations raised certified entanglement as λ increased while test accuracy fell; corrected training failed outright at λ=8.The authors conclude that the surrogate is not established as a regularizer and that a surrogate tracking Δχ under optimization remains open.Source: B Experiment E6 in full: the… · pp. 28–29

C Experiment E7 in full: the interaction spectrum

The interaction spectrum decays with the number of classes, and that decay is stronger in deeper layers. The same sweep supplies a five-class layer summary and scale-domain onset measurements that quantify how higher-order overlap is delayed relative to pairwise interaction.

Campaign scope

  • The spectrum campaign evaluates floors j=2,…,5 on fixed five-class subsets and also examines top floors for all ten classes.The reported deliverables include decay across floors, deep-layer many-class overlap, and depth profiles of onset delays.Source: C Experiment E7 in full: the… · pp. 29, 31

Spectrum decay

  • The mean step ratio b𝐸(Cⱼ₊₁)/b𝐸(Cⱼ) is ≈0.88 at the stem and ≈0.61 at the deepest stage, with monotone decay in every layer of every model.At k=10, top-floor quotients are 0.015–0.07 in deepest ResNet stages versus 0.44–0.73 at the stem; these are magnitude statements because B=19 makes p↓=0.05 the smallest attainable value.Source: C Experiment E7 in full: the… · p. 29, Figure 9 · p. 31

Second-floor summary

  • The second floor of a five-class family ranks layers exactly as the mean of ten pairwise quotients and certifies separation in every layer except two shallow cells.It provides a single layer-level test from the same sweep when testing every pair is unwelcome.Source: C Experiment E7 in full: the… · pp. 29–30

Onsets

  • For the CIFAR-10 ResNet-56 penultimate stage, the 10-fold overlap onset is r*₁₀ ≈ 1.85, versus pairwise onset r*₂ ≈ 0.09.The onset vector stretches monotonically through (0.09, 0.34, 0.64, 0.91, …), while the stem vector is compressed into [0.007, 0.069].Source: C Experiment E7 in full: the… · p. 30, Figure 9 · p. 31

D Experiment E8 in full: the projection dimension

E8 finds that the reported depth-grading structure persists across projection dimensions d0=3, 4, and 6, with deep layers fully certified separated at each dimension. Pair rankings also remain close to the d0=5 reference, supporting the conclusion that the measured shadow’s low dimensionality does not make the reported structure an artifact.

D Experiment E8 in full: the projection dimension

  • The E8 campaign re-measured E2 depth conclusions at d0∈{3,4,6} across three E2 models and pretrained ViT-B/16, comparing 45-pair quotient vectors with the d0=5 reference.Source: D Experiment E8 in full: the… · p. 32
  • At d0=3, 4, and 6, depth grading was unchanged: CIFAR-10 ResNet-56’s mean quotient fell 0.75→0.22, 0.72→0.18, and 0.72→0.16, respectively.The ViT’s non-monotone rise into the penultimate block persisted at every d0, while the pretrained ViT remained monotone throughout.Source: D Experiment E8 in full: the… · p. 32
  • Deep layers were certified separated for 45/45 pairs at every d0, while median Spearman correlations with the d0=5 reference were 0.91, 0.96, and 0.97 at d0=3, 4, and 6.Source: D Experiment E8 in full: the… · p. 32
  • Lower correlations (ρ=0.56–0.69) occurred only at d0=3 where rank noise among near-zero deep quotients or barely moving shallow CIFAR-100 quotients could not change conclusions.Source: D Experiment E8 in full: the… · p. 32

E A coupling we do not claim: grokking

The grokking experiment found that raw profile-mass changes anticorrelated with validation-accuracy changes, reflecting feature-norm shrinkage. The scale-free statistic showed a contemporaneous association in the expected direction, but the authors do not treat it as a finding.

E A coupling we do not claim: grokking

Loading 2609.08561v2…