Routing Probes Can Improve Without New Information: An Exact-Null Audit of Uncertainty Beyond Model Outputs

• The audit separates probe improvement from evidence that routing carries error information beyond outputs, a distinction that matters when routing and outputs are strongly dependent.The paper measures probe comparisons under an exact label null, isolates checkpoint-selection effects, and evaluates corrected comparisons against a conditional reference.Source: §1 · pp. 1, 3
• 308/600 confidence-only MLP evaluations showed a routing gain under the exact label null, while the linear comparison reported none.Across four output views, MLP comparisons with globally and output-matched shuffled routing reported gains in 308 and 293 of 1,920 evaluations, respectively.Source: §1 · p. 1
• 50/120 raw detections fell to 0/120 with validation log-loss selection; an independent probe fell from 83/120 to 0/120.Applying log-loss selection to all 1,920 MLP null evaluations removed all 528 raw detections.Source: §1 · p. 2
• The repaired raw comparison detected an implanted signal of about 0.005 nats in 0/20 replicates in each matched setting; conditional permutation detected it in 11/20 and 10/20.With an estimated conditional law, the conditional test rejected in 59 of 3,840 null evaluations.Source: §1 · p. 2
• Conditional-assignment evidence appeared in 12 of 176 real-label family–probe–view cells, all in five DeiT attention-residual families, and none passed the additional noise criterion.Eight cells retained evidence under two specified conditional-model variants; in the other four, positive seeds fell from two to one.Source: §1 · p. 2

• A fitted probe’s improvement does not establish incremental information: the population information question compares optimal predictors, while a probe searches a restricted function class.The distinction persists with unlimited data and equal nominal parameter counts after zero-padding.Source: §2 · p. 3
• With O uniform on {−1, 0, 1}, R = O^2, and T | O ∼ Bernoulli(σ(O^2)), the probe gain is about 0.0257 nats despite zero incremental information.Here R is a deterministic function of O, and the augmented class contains the Bayes predictor.Source: §2 · p. 3
• A richer baseline can address approximation error but not checkpoint selection; even a better-fitted contrast measures predictive utility, not incremental information.At O0 the output-only logistic class contains the label generator, so the proposition’s approximation route does not explain benchmark gains there.Source: §2 · p. 3

3.1 MATCHED PROBING AND THE SCREENING RULE

• The matched-probe screen compares output-only, real-routing, shuffled-routing, and Gaussian-feature probes using logistic regression or a 64-unit one-hidden-layer MLP.Positive loss differences favor real routing; the pre-specified real-label screen requires positive intervals across seeds for raw, shuffle, and noise components in the same probe and view.Source: §3.1 · pp. 3–4

3.2 A CONDITIONAL REFERENCE FOR THE PROBE STATISTIC

• The conditional permutation test compares the observed routing assignment with assignments drawn from an estimated routing-given-output law; R1 marks repeated unusual assignments, not an estimate of information gain.The test has a guaranteed level only if the estimated law is the true conditional law; R2 is a separate same-width noise criterion, not a requirement for conditional information.Source: §3.2 · pp. 4–5

3.3 AN EXACT LABEL-NULL BENCHMARK

• The exact label-null benchmark freezes a confidence-based logistic correctness generator fitted on disjoint data, keeps real output–routing pairs, and redraws labels for six models.Each probe has 600 evaluations at O0 and O1, 360 at O2 and O3, and 1,920 total; views and probes reuse label vectors, so pooled counts are not independent trials.Source: §3.3 · p. 5

4 RESULTS

• The audit covers 22 routing families, 68 training runs, and 10,000 evaluation examples per run across attention-residual, MoE, and halting models.Configuration snapshots survive for 66 runs, all with a soft-binned calibration penalty; pooled runs are not a controlled comparison of architecture quality.Source: §4 · p. 5

4.1 WHAT THE COMPARISONS REPORT UNDER THE LABEL NULL

• 528/1,920 MLP raw comparisons flag a routing gain under the exact label null, while linear raw comparisons flag none.The benchmark preserves real output–routing pairs while generating labels from an output-only law; every flag is therefore a false positive for incremental information.Source: §4.1 · p. 5

4.2 CHECKPOINT SELECTION PRODUCES THE RAW DETECTIONS

• 50/120 fixed-trajectory flags fall to 0/120 under log-loss selection, and an independent PyTorch MLP reproduces the 83/120-to-0/120 contrast.Accuracy-selected output-only checkpoints have a 0.0264-nat held-out loss gap from the generator, versus 0.0007 under log-loss selection.Source: §4.2 · pp. 5–6
• Applying log-loss selection to all 1,920 MLP evaluations removes all 528 raw detections, while global and matched shuffle detections fall to 27 and 26.Only four conditional-test evaluations are flagged under both selection policies, so the small change in aggregate conditional flags does not imply that individual decisions persist.Source: §4.2 · p. 6, Table 1 · p. 6

4.3 REMOVING NULL DETECTIONS DOES NOT CREATE SENSITIVITY

• 0/20 replicates in each of two settings detect an implanted signal of about 0.005 nats with the repaired raw comparison, while conditional testing detects 11/20 and 10/20.The estimated-law conditional test also flags 0/20 and 1/20 null replicates; raw-comparison operating characteristics do not establish power for the cross-seed R1/R2 rules.Source: §4.3 · pp. 6–7
• The doubled-S sampler check moves 9/16 p-values by more than one grid step and changes one of eight decisions, failing its specified stability check.With the true conditional law, the fixed fitting-and-selection procedure gives exchangeable statistics; estimated-law rejection rates require empirical assessment.Source: §4.3 · p. 7

4.4 CONDITIONAL-ASSIGNMENT EVIDENCE ON REAL CORRECTNESS LABELS

• 12/176 real-label cells show R1, all in five DeiT attention-residual families, but none of the 176 cells meets the additional R2 noise criterion.Eight R1 cells persist under both specified conditional-model variants; in four, positive seeds fall from two to one, and the MLP flags also change with selection policy.Source: §4.4 · p. 7

4.5 THE PRE-SPECIFIED SCREEN ACROSS 22 FAMILIES

• 0/22 families pass the complete screen under either probe, although the output positive control passes in all 22 families.Unadjusted MLP gains reach 0.069 nats, and six family–probe–view combinations pass a standalone shuffle contrast without passing the screen.Source: §4.5 · p. 7

Fitting and selection

• Report both probes’ training loss, validation criterion, and checkpoint rule, and choose one proper selection score in advance.The benchmark measures the cost of mismatched selection: 50/120 against 0/120 detections on fixed trajectories under a known null.Source: §5 · p. 8

Reference, representations, and operating characteristics

• State the reference distribution and assumptions, define what each output view and routing summary retains, and measure the complete decision rule across relevant effect sizes.Shuffling routing globally breaks its output dependence; single-test power does not determine power for a rule combining two scores and requiring at least two positive seeds.Source: §5 · p. 8

Uncertainty from outputs and internal decisions

• Gate entropy has been used for segmentation uncertainty and out-of-distribution detection, tasks that require useful prediction rather than a conditional-information claim.The cited work includes Pavlitska et al. (2025) and Lee et al. (2025).Source: §6 · p. 8

Conditional probing and prior routing audits

• Conditional probing estimates usable information beyond a baseline, and Liang et al. (2026) identify capacity and assignment confounds in attention-residual routing probes.The present additions are measured raw and shuffle behavior under a label null, a fixed-trajectory checkpoint intervention, and sensitivity relative to a conditional reference.Source: §6 · p. 8

Conditional references and predictive importance

• The paper reuses the conditional permutation test of Berrett et al. (2020); its validity depends on a known or well-estimated conditional law of the tested feature.The measurements concern routing-probe comparisons under natural output–routing dependence.Source: §6 · p. 8

Scope

• The scope is in-distribution top-1 correctness for image classification; sorted outputs omit class identity, and routing entropies omit trace detail.Language-model routing, distribution shift, and epistemic–aleatoric decompositions are not studied; coverage of MoE, halting, and external pretrained models is limited.Source: §7 · pp. 8–9

Label null and checkpoint selection

• The label null uses only a logistic function of O0, so it cannot certify the conditional model for other correctness–output dependences.The selection mechanism was isolated at O0 on six models in two MLP implementations; linear probes have no validation-based selection.Source: §7 · p. 9

Conditional-test assumptions and inference

• The conditional test is exact only under the true conditional law, and R1 in four of 12 cells does not persist under two working-model variants.The sampler check fails its one-step criterion on the checked cell, R1/R2 have no multiplicity guarantee, and non-detection cannot establish output sufficiency.Source: §7 · p. 9

8 CONCLUSION

• The conclusion is model-relative: findings establish neither general routing information nor output sufficiency, and interpretation must account for probe fitting, the reference distribution, and the full decision rule.The paper distinguishes fitting a better probe from testing for incremental information.Source: §8 · p. 9

A PRE-SPECIFIED PROTOCOL AND TAXONOMY RULES

• The pre-specified protocol defines gain as finite-probe held-out improvement and requires raw, shuffle, and noise contrasts to meet score and seed conditions for a full pass.Type 0 means the complete criterion was not met; it is not a verdict when the positive control is undetected.Source: A PRE-SPECIFIED PROTOCOL AND TAXONOMY… · p. 12

B SELF-ASSESSMENT AGAINST THE REPORTING STANDARD

• The reporting audit includes common-generator null rates and independent-split power, while warning that component checks do not establish power or validity of the full replicated rule.The output positive control passes in 22/22 families.Source: B SELF-ASSESSMENT AGAINST THE… · p. 13

C PER-FAMILY RESULTS AT EVERY VIEW

• Across all views and both probes, 0/22 families pass the full screen, which requires both scores and seed replication; this does not establish absence of incremental information.Source: Figure 2 · p. 14
• Across all views and both probes, 0/22 families pass the full screen; this does not establish absence of incremental information.The figure plots mean held-out log-loss contrasts, with positive values favoring real routing.Source: Figure 2 · p. 14

D RAW ROUTING-WEIGHT PANEL

• Raw routing weights never outperform the entropy profile in the four-family panel, yet beat Gaussian controls in three families and shuffled routing in none.The comparison is bounded by the fixed-sample setting: the higher-dimensional trace is harder to estimate, and no dimension-controlled study was run.Source: D RAW ROUTING-WEIGHT PANEL · p. 15

E PROOF OF PROPOSITION 1

• With R = O^2, conditional gain is zero for every strictly proper score, yet the augmented logistic class reaches Bayes risk while the output-only class is best constant.The construction uses O uniform on {−1, 0, +1} and T | O ∼ Bernoulli(σ(O^2)).Source: E PROOF OF PROPOSITION 1 · pp. 15–16
• The positive-gap extension covers continuous strictly proper scores with continuous risk on the relevant compact set, but not every strictly proper score.The Brier-score gap is 0.0119; the argument excludes rules with risk unbounded at the boundary.Source: E PROOF OF PROPOSITION 1 · p. 16
• 0.0257 nats of probe gap persists with sufficiently small routing noise despite zero conditional information, so deterministic redundancy is not necessary.The reported gap declines smoothly to 0.0038 nats at δ = 2.Source: E PROOF OF PROPOSITION 1 · p. 16

F MECHANISM DIAGNOSTICS FOR THE TWO HAZARDS

• In a one-cell diagnostic, Gaussian noise, shuffled routing, real routing, and informative output features each degraded held-out score versus their width-matched padded twin.The respective changes were −0.016, −0.025, −0.015, and −0.020 nats; the authors caution this does not establish a universally replicated mechanism.Source: F MECHANISM DIAGNOSTICS FOR THE TWO… · p. 17
• Real routing beat Gaussian noise at 3/3 seeds in three of four matched-panel families but beat shuffled routing in none; a noise-only screen can flag families lacking assignment evidence.In the full sweep, two families passed the noise contrast and failed the shuffle contrast.Source: F MECHANISM DIAGNOSTICS FOR THE TWO… · p. 17

G HISTORICAL FOLD-SPECIFIC LABEL BENCHMARK AND ITS

• The historical fold-specific benchmark showed qualitative agreement on nonlinear-comparator inflation and lower CPT detection, but does not establish T* ⊥ R | O_j.Only the common-generator benchmark draws labels from a single frozen function of O0, making its null exact conditional on that generator.Source: G HISTORICAL FOLD-SPECIFIC LABEL… · p. 18, Table 6 · p. 18

G.1 ORIGINAL FULL-10K BENCHMARK

• The fold-specific construction does not establish an exact conditional null given O_j alone, because identical output views can receive different generator probabilities across folds.The audit found such a case in halting/DeiT/C-100; independence from routing is established only after additionally conditioning on folds and fitted generators.Source: §G.1 · p. 18
• Historical full-sample detections concentrate in the nonlinear probe under the original fold-specific label construction.The benchmark used six pre-specified cells across three backbones, three datasets, and three mechanism classes.Source: §G.1 · p. 18, Table 7 · p. 19

H EXTERNAL PRETRAINED V-MOE VALIDATION

• The external V-MoE findings are descriptive case studies, not identically calibrated procedures, and their synthetic benchmark does not directly calibrate this smaller-effect regime.The external and local audits differ in clipping thresholds and shuffle construction; the external routing check tests only restricted invariance.Source: H EXTERNAL PRETRAINED V-MOE VALIDATION · pp. 19–20
• Neither pretrained V-MoE checkpoint yielded a within-checkpoint three-way pass at any view under either probe (0/8 each).Three of sixteen probe–view combinations had standalone detections, but none formed consistent three-way evidence.Source: H EXTERNAL PRETRAINED V-MOE VALIDATION · p. 19

I HISTORICAL SYNTHETIC COMPONENT-POWER BENCHMARK

• The benchmark injects routing signal at calibrated reference gains from 0 to 0.01 nats and measures raw-gain, shuffle-contrast, and noise-contrast components across sample sizes and routing dimensions.At λ = 0, the construction satisfies T⋆ ⊥ R | O exactly.Source: I HISTORICAL SYNTHETIC COMPONENT-POWER… · p. 20
• At n = 10,000, the paired shuffle contrast reaches 80% power at ≈0.0027–0.0035 nats, while raw gain reaches it at ≈0.003–0.005 nats.At a true gain of 0.002 nats, detection rates are 0.255 vs 0.070 at L = 22 and 0.550 vs 0.445 at L = 8; sensitivity differences depend on routing dimension.Source: I HISTORICAL SYNTHETIC COMPONENT-POWER… · p. 20
• At zero injection, the observed upper bound on joint component-detection rate is 0.000 in all ten null cells, but this is not a confidence bound on the underlying rate.The benchmark stores component rates rather than per-replicate joint indicators, so conjunction power is bounded rather than identified.Source: I HISTORICAL SYNTHETIC COMPONENT-POWER… · p. 20
• The joint component-detection upper bound first reaches 0.8 at 0.0049 nats for L = 22 and 0.0030 nats for L = 8, and these are not family-level power thresholds.The family rule additionally requires agreement across at least two of three seeds.Source: I HISTORICAL SYNTHETIC COMPONENT-POWER… · pp. 20–21
• The positive controls (+0.0037 to +0.0189 nats) lie in the powered regime, but no historical probe–view combination meets the pre-specified three-way routing-specific criterion.Type 0 means the full criterion was not met; the conjunction has Fréchet bounds rather than a calibrated minimum detectable effect, so no verdict asserts an exactly zero effect.Source: I HISTORICAL SYNTHETIC COMPONENT-POWER… · p. 21

J POST-HOC OUTPUT-MATCHED SHUFFLE SENSITIVITY ANALYSIS

• The primary comparison scores each permutation replicate separately; averaging predictions across replicates would give the control an unrelated ensemble advantage under convex log loss and Brier score.The design uses replicate 1 as the primary comparison and the remaining 19 to measure permutation sensitivity.Source: J POST-HOC OUTPUT-MATCHED SHUFFLE… · p. 21
• No matched-shuffle candidate meets the full criterion, and the analysis remains an approximate assignment control rather than an exact conditional test.It standardizes outputs within each fold, groups examples by nearest neighbors, then applies a self-match-free cyclic shift within blocks.Source: Table 8 · p. 21, J POST-HOC OUTPUT-MATCHED SHUFFLE… · pp. 21–22
• MATCHED assignments are more local than GLOBAL assignments at every output view, but locality weakens at higher-dimensional views.Distances are summarized across all 22 families; no examples are unmatched.Source: Table 9 · p. 22, J POST-HOC OUTPUT-MATCHED SHUFFLE… · p. 22

K CHECKPOINT-SELECTION EXPERIMENT

• For the six-model panel, checkpoint selection criterion accounts for the effect: the accuracy- and log-loss-selected arms share a trajectory, so the pooled result describes this panel, not a population of models.Source: K CHECKPOINT-SELECTION EXPERIMENT · p. 23, Table 12 · p. 24

L.1 CONSTRUCTION, EXACTNESS AND IMPLEMENTATION CHECKS

• The exact-null construction freezes a per-model logistic generator fitted on disjoint data, then redraws labels while retaining real output–routing pairs, making routing conditionally uninformative by construction.Six models yield 1,920 model–view–replicate records and 3,840 replicate–probe evaluations; views and probes reuse the same label vectors.Source: §L.1 · p. 27

L.2 COMPLETE CALIBRATION AND HISTORICAL COMPARISON

• The common-generator CPT calibration passes its operational cell gates: 59/3,840 pooled flags, with no model–view–probe cell above 4/100.These empirical gates do not prove exact nominal size for an estimated conditional law or calibrate a replicated family rule.Source: §L.2 · pp. 28–29, Table 20 · p. 29
• Across all output views, common-generator totals are 528/3,840 RAW, 325/3,840 GLOBAL, 305/3,840 MATCHED, and 59/3,840 CPT detections under the original policy.The comparison ordering changes between historical and common-generator benchmarks at O1/MLP and O2/MLP, and these are descriptive comparisons without post-hoc significance tests.Source: §L.2 · pp. 28–29, Table 19 · p. 29

M.1 WORKING CONDITIONAL MODEL AND ASSIGNMENT WEIGHTS

• The conditional analysis estimates a working routing assignment model from output features, and all 272 primary run–view fits select the shared-covariance Q1 tier.Routing coordinates are rank-transformed to normal quantiles; the authors explicitly do not treat the fitted working model as a verified Gaussian law for bounded routing entropies.Source: §M.1 · p. 30

M.2 CONDITIONAL SAMPLER AND PROBE SETTINGS

• The conditional sampler uses reversible heat-bath swaps, with 50 rounds to form a hub and 79 spokes of 50 rounds each; exchangeability is guaranteed only with the true conditional law.Movement diagnostics do not certify fidelity of an estimated law.Source: §M.2 · p. 30

M.3 R1, SAME-HALF R2 AND SEED REPLICATION

• R1 requires both primary p-values to pass within a seed and positive results in at least two seeds per family–probe–view cell; R2 additionally requires same-width noise positivity.The analysis assigns no multiplicity-adjusted familywise interpretation to R1 or R2, and R2 is a robustness criterion rather than a necessary condition for conditional information.Source: §M.3 · p. 30

M.4 HISTORICAL FOLD-SPECIFIC PRIMARY CALIBRATION

• Historical CPT calibration yields 74/3,840 pooled flags, with view–probe rates from 4/360 to 9/360, all within their pre-specified operational bands.Only two R1-positive families have directly corresponding calibration settings, and those cover training seed 0 only.Source: §M.4 · p. 31, Table 21 · p. 31, Table 22 · p. 31

M.5 OBSERVED-CORRECTNESS CONDITIONAL EVIDENCE AND DIAGNOSTICS

• 12/176 R1-positive cells occur in five of 22 families, all among the six DeiT attention-residual families; the other 16 families have no positive cells.Ten detections use linear probes and two use MLPs; cell means span 0.001221–0.005424 nats, while individual-seed effects include negative values.Source: §M.5 · pp. 31–32, Table 23 · p. 32, Table 29 · p. 36
• No R1-positive seed/view triggers the pre-specified F1 warning, but overlapping diagnostic ranges and other q-model problems remain possible.The passage explicitly cautions that these diagnostics do not rule out other working-model problems and that the noted fit concern is not evidence of causality.Source: §M.5 · p. 32
• The doubled-S check fails its pre-specified criterion in one cell per view, and its scope does not establish effects on other tests.At S = 100, 9 of 16 p-values move by more than one grid step; the same seed streams mean these shifts are not independent Monte Carlo replicates.Source: §M.5 · p. 33, Table 25 · p. 33

M.6 WORKING-MODEL SENSITIVITY OF THE R1 CELLS

• Eight of the 12 R1-positive cells retain R1 under both specified conditional-model variants, while the same four lose it under both.Q2 and Q3 fit less well than Q1, so this agreement establishes stability to the specified variants, not that any model is the true conditional law.Source: §M.6 · p. 34, Table 26 · p. 34

M.7 INDEPENDENT-SPLIT LINEAR POWER ENDPOINT

• Each of the two independent-split linear power cells reaches 200/200 empirical detections at the 0.010-nat target, but this does not establish MLP or smaller-effect power.The endpoint does not establish the full power curve or the power of a replicated CPT-plus-noise conjunction.Source: §M.7 · p. 35, Table 27 · p. 35

M.8 ALL SAME-HALF NOISE COMPARISONS AND LOGICAL COMPLETION

• 0/176 cells meet the final same-half R2 criterion, despite three joint-positive seed comparisons in different cells that do not replicate within their cells.All other cells already had fewer than two CPT-positive seeds, so their R2 verdicts followed by logical short-circuiting.Source: §M.8 · p. 35, Table 28 · p. 35

N LOCAL MODEL CONFIGURATION AND EVALUATION DATA

• Attention-residual models use learned softmax weights over prior hidden states, while local MoE and halting models use a DeiT-Small-shaped trunk and specified routing traces.Pooled historical runs are not a controlled architecture-quality comparison; the reused attention-residual checkpoints are not an independently trained replication.Source: N LOCAL MODEL CONFIGURATION AND… · p. 38, Table 31 · p. 39

O.1 ORIGINAL AUDIT CORRECTIONS

• Historical intermediate readings are not current scientific conclusions; the corrected evidence and remaining limitations appear in the scientific appendices.Source: O REPRODUCIBILITY AND PROVENANCE · p. 39, §O.1 · p. 40, Table 32 · p. 40

O.2 CONDITIONAL-ANALYSIS CORRECTIONS

• The repair’s operational verdict does not establish validity under the true conditional law or remove the qualification of the historical generator.It supports only the statement that no evaluated cell meets the same-width robustness criterion.Source: §O.2 · pp. 40–41

O.3 COMMON-GENERATOR COMPUTATIONAL PROVENANCE

• The common-generator benchmark retained inflated MLP shuffle-control detection rates, while linear comparisons had low rates for all controls.The pre-specified interpretation also records that control ordering changed between settings.Source: §O.3 · p. 41

O.4 DETERMINISTIC RANDOM STREAMS

• The appendix specifies distinct seed rules for label, calibration, power, and conditional analyses, including a fixed split formed from a seeded permutation.Source: §O.4 · p. 41

O.5 IMMUTABLE ARTIFACT IDENTIFIERS

• Artifact identifiers locate result and interpretation records, while hashes identify bytes rather than statistical validity.Source: §O.5 · p. 41, Table 33 · p. 42

O.6 SEED AUDIT

• Shared control streams add dependence to pooled comparator counts, so those rates are descriptive rather than independent Bernoulli evidence.The two null benchmarks also share permutations, making the second a paired re-analysis rather than independent corroboration.Source: §O.6 · pp. 43–44

O REPRODUCIBILITY AND PROVENANCE

• The provenance audit records campaign policies and pairing checks, but producer identity is reconstructed from logs and sampler agreement does not prove bitwise assignment identity.The two null benchmarks’ assignment arrays were separately recomputed and matched by SHA-256.Source: §O.6 · p. 44