PaperlayerView exact revision on arXiv

Source-linked AI summary

Auditing Routing Entropy as an Uncertainty Signal in Attention-Residual Transformers

Wenhao Liang, Lin Yue, Wei Emma Zhang, Mingyu Guo, Olaf Maennel, Weitong Chen

arXiv:2610.01495v1cs.AI

TL;DR

Dynamic models leave per-example routing traces, but whether those traces add correctness information beyond confidence is an open question. This paper tests AR routing entropy with conditional, replicated, controlled-utility checks and injected-signal audits; profiles beat shuffled traces but not confidence-only prediction, and simulations show incomplete recovery.

  • Problem

    The paper asks whether routing traces signal correctness beyond the model’s confidence, since a trace associated with errors may add no information the prediction does not already reveal.

    Source: §1 · p. 1

  • Method

    The study tests AR routing entropy through fixed-confidence diagnostics, seed replication, held-out comparisons with output-only and shuffled-trace controls, and injected-effect sensitivity audits.

    Source: §1 · p. 2, §3 · p. 4

  • Results

    Across 24 paired runs, entropy profiles beat shuffled profiles but lost to confidence-only prediction; apparent binned hits did not replicate, and probes recovered only part of injected effects.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Evaluations of internal traces as uncertainty signals should report output-only comparisons, seed replication, and recovery of specified injected signals before interpreting a null.

    Source: §6 · p. 9

    Evidence is limited to small, from-scratch CIFAR-10/100 models with a calibration auxiliary loss and four replicated backbone–dataset settings.

    Source: §6 · p. 9

Abstract

from arXiv · show

Dynamic architectures leave a per-example routing trace beside each prediction, and diffuse routing is easy to read as a sign that the prediction is unreliable. We audit that reading for routing entropy in Attention-Residual (AR) variants of Swin-Tiny and DeiT-Small, trained from scratch on CIFAR-10/100 with a soft-binned calibration auxiliary loss, asking whether the trace carries information about correctness beyond what the model's own confidence already reveals. Three checks probe this increment: does a routing signal appear at fixed confidence, does it replicate across training seeds, and can a held-out predictor exploit it against output-only and shuffled-trace controls? A sensitivity audit then injects effects of known size and measures the fraction of each that the probes recover. No test in the fixed 30-test binned family survives multiplicity correction, and neither the nominal hit nor a borderline result recurs in its sibling seeds. Across 24 paired runs a scalar routing probe yields no pooled improvement in routing-stratified calibration, and an entropy-profile probe predicts correctness better than the same probe given shuffled profiles yet worse than a confidence-only predictor in both binary log-loss and Brier score: a gain over shuffled traces does not become a gain over the output. Conditioning on the complete logit vector leaves the corresponding comparison unresolved. The audit bounds how far these non-detections can be read: at an injected effect of 0.010 nats the profile probe recovers 24-59% of the oracle gain, and a reference-preserving correction probe recovers 8% and 23% in the two CIFAR-100 settings, below the threshold we fixed for applying it to real labels. The results establish control-dependent gains and incomplete estimator recovery, not the absence of conditional routing information.

1 INTRODUCTION

Dynamic architectures leave per-example routing traces, prompting the question of whether routing entropy signals prediction reliability beyond confidence. In AR models, the reported gains depend on the control, apparent binned effects do not replicate, and sensitivity audits limit what non-detections establish.

1 INTRODUCTION

  • Per-example traces arise in mixture-of-experts, adaptive-depth, and token-pruning models, motivating tests of whether those traces signal unreliable predictions [Shazeer et al., 2017; Fedus et al., 2022; Wang et al., 2018; Rao et al., 2021].The paper presents its contribution as new evidence about uncertainty claims for internal routing, using established conditional-probing and oracle-score tools [Hewitt et al., 2021; Xu et al., 2020].Source: §1 · pp. 1, 3
  • The central question is whether routing changes correctness probability among predictions with equal confidence, rather than merely associating with errors.The primary analyses condition on top-class confidence; a secondary analysis conditions on the complete logit vector.Source: §1 · p. 1
  • Three checks test conditional signal, replication across seeds, and held-out utility against output-only and shuffled-trace controls, while an injected-effect audit measures probe recovery.The checks address residual confounding from coarse confidence bins, seed stability, and whether a flexible probe gains from an input that adds no new correctness information.Source: §1 · p. 2
  • Across 24 paired runs, entropy profiles beat shuffled profiles but not confidence-only prediction; apparent binned hits did not replicate, and the complete-logit comparison remained unresolved.The sensitivity audit also found that probes recovered only part of an injected routing effect.Source: §1 · p. 2

2 ROUTING INFORMATION BEYOND CONFIDENCE

The section defines correctness information beyond confidence, represents routing with entropy profiles and scalar summaries, and specifies conditional, controlled-utility, and sensitivity checks alongside the probes used to evaluate them.

2.1 TARGET AND ROUTING REPRESENTATION

  • An oracle’s gain from observing confidence and routing over confidence alone equals E[(ηc,R − ηc)^2] in binary Brier risk and I(T; R | c) in binary log-loss.Fitted predictors can fall short of these oracle gains.Source: §2.1 · p. 3
  • At each AR layer, normalized routing entropy averaged over tokens forms a profile; aggregate entropy defines diagnostic groups, while depth variation feeds the scalar calibration probe.The profile summarizes rather than retains raw routing weights; layers with only one available state are omitted.Source: §2.1 · p. 3

2.2 THREE CHECKS AND A SENSITIVITY AUDIT

  • The conditional diagnostic compares accuracy in low- and high-aggregate-entropy tertiles within confidence bins, with a continuous-confidence check using cross-fitted logistic models and local-window permutations.The binned test uses 5,000 within-bin permutations; the multiplicity family is fixed at 30 tests, including 24 replicated runs and six single-seed ViT-B/16 runs.Source: §2.2 · p. 3
  • Held-out routing probes are compared with same-family output-only predictors and, for profiles, shuffled traces; shuffling preserves P(R) but not P(R | c).Zero-padding matches nominal parameterization but not effective capacity, and a profile probe can beat shuffled profiles without adding information beyond confidence.Source: §2.2 · p. 3
  • The sensitivity audit injects specified routing effects into simulated labels and measures fitted gain divided by oracle gain, rather than test rejection rate.Its alternative label law preserves empirical marginals while smoothing their joint distribution; the profile-probe screen required at least 25% recovery at 0.010 nats before weaker strengths or larger simulations were run.Source: §2.2 · pp. 3–4

2.3 PROBES

  • The scalar probe estimates correctness from confidence and depth variation with a kernel regression, while the profile probe predicts correctness from confidence and layerwise entropies.The profile, confidence-only, and shuffled arms use inner-validation selection between a one-hidden-layer MLP and shared ridge-logistic model, with outer cross-fitting.Source: §2.3 · p. 4

3 EXPERIMENTAL SETUP

The experiments use replicated AR variants of Swin-Tiny and DeiT-Small on CIFAR-10/100, with held-out analyses and diagnostic tests organized across runs and backbone–dataset settings.

3 EXPERIMENTAL SETUP

  • The replicated design covers 24 runs across two backbones, two datasets, two AR variants, and three seeds; six single-seed ViT-B/16 runs enter only the diagnostic family.Models were trained from scratch with the DeiT recipe and a soft-binned ECE auxiliary loss; three CIFAR-10 seed-0 runs deviated in batch size or schedule.Source: §3 · p. 4
  • The diagnostic uses all 10,000 test examples, scalar calibration uses a fixed 5,000/5,000 split, and profile and complete-output probes cross-fit over all 10,000.Pooled intervals use a hierarchical bootstrap over four backbone–dataset settings, so the 24 runs are not 24 independent settings.Source: §3 · p. 4
  • Figure 2 displays within-bin permutation p-values for the fixed 30-test aggregate-entropy family, distinguishing replicated Swin/DeiT runs from single-seed ViT runs and marking 0.05 and 0.05/30 thresholds.The star marks the nominal hit and the diamond the borderline run.Source: Figure 2 · p. 5

4 RESULTS

Across the tested settings, routing signals did not yield replicated, multiplicity-corrected evidence of conditional correctness information or a pooled scalar-calibration improvement. Entropy profiles beat shuffled traces but not confidence-only prediction; full-logit comparisons remain unresolved, and injected-effect audits show incomplete recovery.

4.1 CONDITIONAL SIGNAL AND REPLICATION

  • Neither apparent binned routing effect replicated in sibling seeds, and no test passed the 0.05/30 = 0.0017 threshold.Continuous-confidence checks across all 24 replicated runs found no consistent routing improvement; the one whole-profile gain did not recur in its sibling seeds.Source: §4.1 · p. 5

4.2 SCALAR ROUTING AND CALIBRATION

  • Across 24 paired runs, the scalar probe minus NW(c) had pooled worst-tertile ECE +0.00038 [−0.00168, +0.00245], with no pooled improvement.Calibrated class probabilities were also worse with routing in multiclass NLL, +0.0084 [+0.0061, +0.0109], and Brier score, +0.0010 [+0.0008, +0.0013].Source: §4.2 · pp. 5–6, Figure 3 · p. 6
  • Scalar-probe simulations recovered 54–65% of a 0.010-nat effect, but two-dimensional estimation remained worse than NW(c) at injected effects of 0.0005 and 0.002 nats.These simulations suggest an estimator cost under the specified null laws but do not isolate the cause of the real-data cost; compressing the profile may discard layer-wise information.Source: §4.2 · p. 6

4.3 THE ENTROPY PROFILE: TWO CONTROLS, TWO ANSWERS

  • Profile minus confidence-only was +1.28 [ +0.45, +2.36] in binary log-loss and +0.32 [+0.13, +0.54] in Brier score, while profile beat shuffled profiles.Both orderings held in every setting mean and leave-one-setting-out mean; the shuffle breaks dependence between routing and confidence, so it is not a conditional null.Source: §4.3 · p. 6, Figure 4 · p. 7

4.5 ROBUSTNESS

  • At 0.010-nat injected effects, the profile probe recovered 24–59% of the oracle gain, while the reference-preserving correction probe recovered 8.1% and 22.6% on the two CIFAR-100 settings.The correction probe fell below its 25% continuation threshold, so it was not evaluated on real correctness labels; robustness checks also found metric rankings depended on ECE choice.Source: §4.3 · p. 6, §4.4 · p. 7, §4.5 · p. 7

4.4 CONDITIONING ON THE COMPLETE OUTPUT

  • With complete-output conditioning, profile minus output-only remained unresolved: +0.87 [−0.05, +2.46] under selected learners and +0.21 [−0.03, +0.46] with ridge-logistic learners.The profile beat permuted profiles, but CIFAR-100 selections returned the output-only reference throughout; extending the MLP grid left the selected contrast interval spanning zero.Source: §4.4 · pp. 6–7, Figure 4 · p. 7

5 RELATED WORK

The related work spans post-hoc calibration, representation-based failure prediction, probing, subgroup calibration, and dynamic computation. This paper distinguishes its question by testing whether routing adds correctness information after confidence is known.

5 RELATED WORK

  • Prior failure-prediction approaches use post-hoc calibration or hidden representations, including Trust Score, ConfidNet, and language-model probes; this work asks about routing beyond confidence.Post-hoc calibration maps outputs to probabilities (Guo et al., 2017), while the cited representation methods predict correctness or related outcomes.Source: §5 · p. 8
  • Controlled probing compares a baseline-plus-representation predictor with a baseline-only control, while conditional-permutation guarantees depend on the reference distribution.The output-only arm uses the model output as baseline and correctness as target; the global shuffle is a trace-destruction comparison, not a conditional null.Source: §5 · p. 8
  • The routing-defined tertile score tests one specified subgroup partition, while ECE rankings can vary by metric and dynamic-computation work does not claim conditional uncertainty.AR extends input-dependent selection to residual aggregation (Kimi Team et al., 2026), in work focused on efficiency and accuracy.Source: §5 · p. 8

6 DISCUSSION

The profile probe beats shuffled traces but not confidence-only prediction, and incomplete recovery of known routing effects prevents interpreting that comparison as evidence that routing carries no information. The appendix defines worst-tertile ECE as a stress test of calibration heterogeneity across routing-defined groups.

6 DISCUSSION

  • The profile probe improves on shuffled traces but not confidence-only prediction, and the results do not establish whether routing information is absent or incompletely recovered.The authors recommend output-only comparisons, seed replication, and sensitivity checks before interpreting null results; transfer to other traces is suggested but untested.Source: §6 · pp. 8–9
  • The evidence is limited to small models trained from scratch on CIFAR-10/100 with a calibration auxiliary loss; larger pretrained models and other settings remain unstudied.The correction probe was not run on real correctness labels, and the conclusions need not carry over to cross-entropy-only AR models.Source: §6 · p. 9

A WORST-TERTILE ECE: BIN-LEVEL CONSTRUCTION

  • The tertile-wise ECE uses 10 confidence bins, right-closed so confidence 1 falls in the last bin; tertiles with fewer than 10 examples would be skipped, though none are.Ties follow the stated strict-inequality convention, and the worst tertile's error is reported.Source: A WORST-TERTILE ECE: BIN-LEVEL… · p. 12
  • Worst-tertile ECE measures calibration error in the most miscalibrated routing-defined subgroup, rather than the global average, to stress-test heterogeneity along the scalar-probe feature.The evaluation set is split into low, middle, and high routing-score tertiles, with equal-width confidence bins computed separately within each tertile.Source: A WORST-TERTILE ECE: BIN-LEVEL… · p. 12

B SINGLE-RUN DIAGNOSTIC AND DESCRIPTIVE CALIBRATION BENCHMARK

The single-run diagnostic and calibration benchmark are descriptive, with a nominal diagnostic hit in the 30-test family. Reliability diagrams show under-confidence in the uncalibrated Block-AR panels, while calibration results vary by substrate.

B SINGLE-RUN DIAGNOSTIC AND DESCRIPTIVE CALIBRATION BENCHMARK

  • The nominally positive single-run diagnostic remains visible in the table and cross-run figure, but the supplied passages give no cell values or inferential conclusion for this result.The diagnostic uses the full 10,000-example test set; the calibration benchmark fits methods on a 5,000-example calibration half and evaluates on a held-out 5,000-example half.Source: B SINGLE-RUN DIAGNOSTIC AND… · p. 12, Table 2 · p. 13
  • The 14-method benchmark covers Block-AR b=2 Swin-Tiny/CIFAR-10 across three seeds, but its table caption supplies scope and metric conventions rather than method scores.The catalogue includes external baselines and three controlled NW variants; the worst-tertile ECE uses rstd.Source: B SINGLE-RUN DIAGNOSTIC AND… · p. 12, Table 3 · p. 13

C RELIABILITY DIAGRAMS ON BLOCK ATTENTION RESIDUAL

  • Uncalibrated Block-AR panels are under-confident, and all three calibrators reduce the gap; AR-CondCal has lower global-ECE point estimates on the two CIFAR-100 cells but is not uniformly superior across substrates.The panels use seed-0 checkpoints and a 5,000-example test half; the diagrams are descriptive, not primary-result inputs.Source: C RELIABILITY DIAGRAMS ON BLOCK… · p. 13, Figure 5 · p. 14

D CALIBRATION-SET-SIZE SENSITIVITY

On the examined Swin-Tiny/CIFAR-10 Block-AR run, Temperature Scaling is more sample-efficient than the two-dimensional methods, while alternative ECE metrics change method rankings. Paired proper-score results favor Vector Scaling over AR-CondCal.

D CALIBRATION-SET-SIZE SENSITIVITY

  • At ncal=250, Temperature Scaling ECE is 0.0244 versus 0.0106 at ncal=5000, a factor of ≈2.3; MEC degrades by ≈3.4 and AR-CondCal by ≈3.0 over the same range.AR-CondCal is not more sample-robust than MEC on this run; at ncal ≤1000 their mean curves overlap within the reported variability bands.Source: D CALIBRATION-SET-SIZE SENSITIVITY · p. 14, Figure 6 · p. 15

E MCE, CLASSWISE ECE, AND SMOOTHECE (COMPLEMENT TO TABLE 3)

F PAIRED NLL AND BRIER DELTAS VS UNCALIBRATED MODEL

  • Vector Scaling gives the strongest paired improvement on both proper scores, while AR-CondCal achieves ∆NLL = −0.1202 ± 0.0056, a 0.0185 smaller mean improvement than Vector Scaling.The comparison uses paired changes from each seed's uncalibrated score across three Block-AR b=2 Swin-Tiny/CIFAR-10 seeds.Source: F PAIRED NLL AND BRIER DELTAS VS… · p. 16, Table 5 · p. 16

G ROBUSTNESS CHECKS FOR THE ROUTING-CONDITIONAL GAP

The routing-conditional checks found no replicated binned result, and the pooled scalar probe did not improve worst-tertile ECE. Profile-probe gains depended on the control, while sensitivity audits showed incomplete recovery of injected effects.

G ROBUSTNESS CHECKS FOR THE ROUTING-CONDITIONAL GAP

  • The p = 0.0382 Block-AR hit (siblings 0.4771, 0.6989) and p = 0.0494 borderline run (siblings 0.9268, 0.7471) were unreplicated.The binned statistic approximates fixed-confidence comparisons and is sensitive to finite support.Source: G ROBUSTNESS CHECKS FOR THE… · pp. 16–17
  • Null simulations found a largest Type-I estimate of 0.0545 at nominal 0.05; continuous-confidence and local-caliper checks yielded no corrected, seed-consistent routing signal.The continuous-confidence profile improvement was uncorrected and did not recur in sibling seeds; local-caliper tests had 5 nominal hits among 96 comparisons, versus 4.8 expected.Source: G ROBUSTNESS CHECKS FOR THE… · p. 17

H BANDWIDTH SENSITIVITY FOR AR-CONDCAL

  • Across eight selected CIFAR-10 runs, no bandwidth multiplier was consistently best; CV-NLL and oracle-ECE shifts (+0.0009 and −0.0003) were within the cross-run standard deviation of approximately 0.007.The evaluated audit does not cover anisotropic grids, non-Gaussian kernels, or objectives that directly tune worst-tertile ECE.Source: H BANDWIDTH SENSITIVITY FOR AR-CONDCAL · pp. 18–19

I POOLED EFFECT FOR THE SCALAR PROBE

  • The pooled scalar-probe WTECE difference was +0.000380 (95% CI [−0.001677, +0.002449]), without a formal equivalence claim across four settings.Proper-score guardrails favored confidence-only: NLL +0.008423 ([+0.006055, +0.010903]) and Brier +0.001031 ([+0.000760, +0.001341]).Source: I POOLED EFFECT FOR THE SCALAR PROBE · p. 19

J ENTROPY-PROFILE CORRECTNESS PREDICTION

  • Profile-minus-confidence-only log-loss remained positive after omitting any one setting, while profile-minus-shuffled differences stayed negative in both log-loss and Brier score.The pooled analysis paired the profile, confidence-only, and shuffled arms across 24 runs; the four observed settings support procedure-specific intervals.Source: J ENTROPY-PROFILE CORRECTNESS PREDICTION · p. 20

J.1 AN EARLIER IMPLEMENTATION OF THE PROFILE PROBE

  • On the same 23 runs, the revised procedure changed profile-minus-confidence log-loss from −0.0199 to +0.0013; the earlier procedure is excluded from the conclusions.The earlier implementation used accuracy-based stopping, all-example feature scaling, and shuffle donors across outer-fold boundaries.Source: §J.1 · pp. 20–21

J.2 SENSITIVITY AUDIT OF THE SCALAR AND PROFILE PROBES

  • At 0.010 nats, the binned test's family-threshold power ranged from 0.185–0.85 by setting, while the scalar NW probe recovered 54–65% of the effect.The binned diagnostic and NW probe use different routing teachers; rejection rates and recovery are distinct quantities.Source: §J.2 · p. 21, Table 11 · p. 22
  • DeiT/CIFAR-100 failed the prespecified 0.005-nat screen (mean gain −0.000540), stopping weaker simulations; at 0.010 nats, it recovered 24.4%.The screen required positive lower Monte Carlo bounds at 0.005 and 0.010 nats and at least 25% mean recovery at 0.010 nats.Source: §J.2 · p. 21, Table 12 · p. 22

J.3 CONDITIONING ON THE COMPLETE OUTPUT

With the complete output as the conditioning set, the selected and ridge profile comparisons against output-only predictors remain unresolved, although profiles outperform shuffled profiles. The control and sensitivity analyses show that this result depends on the fitted learners and the injected signal being tested.

J.3 CONDITIONING ON THE COMPLETE OUTPUT

  • Against output-only predictors, selected and ridge profile comparisons had intervals containing zero, while profile-versus-permuted comparisons excluded zero in both learners.The MLP-arm profile-minus-output-only interval excluded zero, but that was the arm most often rejected by the selection rule.Source: §J.3 · p. 24
  • On K=100 runs, all selected arms fell back to the shared reference, making the selected-model difference exactly zero by construction.Across runs, the routing-augmented network probe was worse than its output-only counterpart by 1.377 × 10−3 nats and worse than the ridge reference by 3.823 × 10−3.Source: Table 13 · p. 23, §J.3 · p. 24
  • Extending the MLP weight-decay grid reduced fallbacks from 563 to 407 and brought the output-only MLP to the reference average, but learner adequacy at the grid edge remained open.The extended-grid selected contrast was +0.35 [−0.18, +1.20], and the largest weight-decay value was selected in 636 selections.Source: §J.3 · p. 24, Table 14 · p. 25
  • Under the injected-signal law Q, the oracle gain matched the c-conditioned oracle gain to eight decimals in all four caches.Because qQ(z) = q0(c), no measurable function of Z alone can improve on q0 under Q.Source: §J.3 · p. 25
  • At 0.010 nats, the injected routing gain closed 21.8% and 57.4% of the output-only probe's deficit in the two CIFAR-100 settings, yet the augmented candidate remained behind the reference.The control concerns the fitted MLPs in these settings, not output dimensionality, which was not ablated.Source: §J.3 · p. 25
  • The positive control tests profile signal orthogonal to Z given c; it does not test interactions between output and routing, and one teacher cannot validate all interaction signals.A recovered-gain fraction is neither a detection probability nor statistical power at a fixed test level.Source: §J.3 · p. 25

K SUPPORTING PROPOSITIONS

The propositions define routing’s oracle value beyond confidence, motivate subgroup comparisons, and explain why finite-sample probes may fail to recover that value. The section also describes MEC as an internal logits-only comparator.

K SUPPORTING PROPOSITIONS

  • Adding routing to confidence improves oracle Brier and log-loss exactly when routing changes conditional correctness on a positive-mass set.The gains equal the squared difference between conditional correctness means for Brier and conditional mutual information for log-loss.Source: K SUPPORTING PROPOSITIONS · p. 26
  • The oracle identities concern population Brier and log-loss, whereas ECE and the fitted probes are finite-sample quantities whose ordering can depend on feature range, bandwidth, and sample size.They make no claim about baseline dominance, accuracy, robustness, or safety.Source: K SUPPORTING PROPOSITIONS · pp. 26, 28
  • Where routing subgroups have different conditional accuracies at the same confidence, no confidence-only calibrator can eliminate the worst-subgroup mismatch.The empirical max-over-bins statistic motivates this test but is not a direct estimate of the pointwise population bound.Source: K SUPPORTING PROPOSITIONS · pp. 26–27
  • Binning confidence can leave within-bin confounding, while sparse support and maximising over bins can increase variance and complicate inference.With 30 tests, 1.5 nominal rejections are expected under exactly uniform null p-values at α = 0.05; this does not test the joint null.Source: K SUPPORTING PROPOSITIONS · p. 27
  • Kernel regression is consistent only under stated regularity conditions, with no finite-sample guarantee for this study’s calibration sizes.In the stylised two-group calculation, the retained subgroup gap shrinks toward zero as bandwidth grows relative to subgroup spacing; this is explanatory, not an empirical attribution.Source: K SUPPORTING PROPOSITIONS · pp. 27–28

L INTERNAL COMPARATOR: MEC

  • MEC is an internal logits-only comparator using margin, predictive entropy, and optionally predicted class, with conditional class offsets and quantile-binned log-temperature adjustments.It first fits a global temperature-scaling anchor; no monotonicity claim is made.Source: L INTERNAL COMPARATOR: MEC · p. 28

M CROSS-ARCHITECTURE AND MULTI-SEED EVIDENCE

The diagnostic family includes replicated Swin-Tiny and DeiT-Small settings and descriptive single-seed ViT-B/16 runs. The reported binned hits do not survive multiplicity correction or recur in sibling seeds, while matched routing profiles are descriptive rather than inferential.

M CROSS-ARCHITECTURE AND MULTI-SEED EVIDENCE

M.1 THE 30-TEST DIAGNOSTIC FAMILY

  • In the 30-test diagnostic family, every value lies far above 0.05/30; restricting the family to 24 replicated runs at threshold 0.05/24 leaves the multiplicity conclusion unchanged.The borderline seed-1 run has p = 0.0494 with 5,000 permutations and p-values from 0.0479–0.0503 in five recomputations with 50,000 permutations.Source: §M.1 · p. 29

N ROUTING-FEATURE GEOMETRY AND LAYER-WISE STRUCTURE

  • After confidence matching, correct and incorrect routing-entropy profiles look visually similar in better-supported bands, but the plots do not test conditional independence.High-confidence rows have fewer than 50 pairs and are marked low support.Source: Figure 7 · p. 30

N.1 SCALAR ROUTING-FEATURE GEOMETRY AND ABLATION

Scalar routing summaries show substantial overlap and no consistent advantage, while marginal associations do not establish conditional information. The section records the study’s training, calibration, and probe protocols and cautions that profile probes test particular learners.

N.1 SCALAR ROUTING-FEATURE GEOMETRY AND ABLATION

  • The scalar rstd distributions overlap almost completely for correct and incorrect predictions, with conditional means 0.1769 and 0.1790.The depth comparison reports 8 Block-AR versus 16 Full-AR sub-layers, with cross-depth standard deviations 0.171 and 0.142, respectively.Source: §N.1 · p. 30

N.2 EXTENDED DISCUSSION OF §6

  • The marginal rstd association with absolute correctness residual is not conditional correctness, and no conditional increment follows from it.Across the matched settings, layer-wise differences are small and setting-dependent without a visually common depth-localised pattern; the profile probe also loses to confidence-only prediction.Source: §N.1 · p. 31, Figure 8 · p. 31, §N.2 · pp. 31–32

O TRAINING AND EVALUATION DETAILS

P IMPLEMENTATION NOTES

  • Routing caches were extracted without retraining, and label-free min–max rescaling left matched-kernel NW results unchanged.The main AR-CondCal row uses Scott’s-rule bandwidths.Source: P IMPLEMENTATION NOTES · p. 33

Q REPRODUCIBILITY PROTOCOL SUMMARY

  • The profile probe uses cross-fitting, selects between an MLP and confidence-only logistic reference, and evaluates binary log-loss and Brier score.Its sensitivity audit stopped at 0.010 nats under a pre-fixed rule, so weaker profile strengths were not tested.Source: Q REPRODUCIBILITY PROTOCOL SUMMARY · p. 34

R KEEPING THE REFERENCE: A RESIDUAL CORRECTION PROBE

A follow-up keeps a fitted reference predictor and tests whether a routing-based residual correction improves it in simulations. The correction recovers a positive but limited fraction of injected oracle gain, and the simulation result does not establish the increment on real labels.

R KEEPING THE REFERENCE: A RESIDUAL CORRECTION PROBE

  • The probe fits a shared reference, then learns output-only, routing, or shuffled-routing log-odds corrections with matched capacity and search budget.All labels are simulated; the shuffled-routing arm is a destruction control, not an exact conditional-null sampler.Source: R KEEPING THE REFERENCE: A RESIDUAL… · p. 34
  • The paired follow-up reuses previously observed simulation records, so it is not a fresh independent validation.Phase 1 covered two CIFAR-100 settings at 0, 0.005, and 0.010 nats with 20 replicates per cell.Source: R KEEPING THE REFERENCE: A RESIDUAL… · p. 34
  • At 0.010 nats, the correction recovers 8.14% on Swin/CIFAR-100 and 22.57% on DeiT/CIFAR-100, below the pre-fixed 25% mean-recovery criterion.The cross-setting mean recovery is 15.35%; the criterion was missed both on the recorded mean and the added per-setting check.Source: R KEEPING THE REFERENCE: A RESIDUAL… · p. 35
  • The selected correction’s null loss disadvantage is not equivalence, and the follow-up does not establish routing information on real correctness labels.The reported movement against the earlier candidate includes baseline predictive improvement; the same-budget paired increment measures routing recovery.Source: R KEEPING THE REFERENCE: A RESIDUAL… · p. 35

S RUNS, CODE AND DATA

The archive supports reproduction of the analyses on the 24 replicated runs from cached routing data and includes code, per-run results, and a manifest linking headline numbers to their sources. The run records also document cache checks and retain two schedule-deviating seed-0 runs under the fixed inclusion rule.

Run and cache integrity

  • The 24 replicated runs each have a distinct-logit cache, and cache accuracy matches the training-log final accuracy within 0.0002.A duplicate cache under a tag outside the design was found and excluded.Source: S RUNS, CODE AND DATA · p. 35
  • Two lower-accuracy seed-0 runs—Swin-Tiny Full-AR at 0.744 and DeiT-Small Block-AR at 0.695—remain included under the fixed rule despite schedule deviations.Both reached epoch 299; the post-hoc sensitivity analyses omit them, alone or with a third deviating run.Source: S RUNS, CODE AND DATA · p. 35, Table 18 · p. 36

Reproducibility levels

  • The benchmark, diagnostic, and probes read cached logits, labels, and per-layer routing entropies extracted from checkpoints.The archive contains the 24 replicated-run caches and analysis code, but not model checkpoints; retraining or re-extracting caches therefore requires training runs or checkpoints.Source: S RUNS, CODE AND DATA · p. 36
  • One command reproduces pooled summaries, the 30-test diagnostic, Tables 1, 6, and 10–14, and Figs. 2–4 from stored results; cache-based scripts regenerate Figs. 5–9.In the reference environment, recomputed files were byte-identical and figures pixel-identical; cache refits cover analyses of the 24 replicated runs, but not the six ViT-B/16 diagnostic values.Source: S RUNS, CODE AND DATA · p. 36
Loading 2610.01495v1…