PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond the Trace: Coupling an Interpretable Reasoning-State Readout to Native MoE Routing

Kang Chen, Sihan Zhao, Yixin Cao, Yugang Jiang

arXiv:2608.17638v1cs.AI

TL;DR

Reasoning traces expose emitted text but not the full latent process state needed for earlier selection and compute-allocation decisions. This paper introduces J64, a readable semantic readout, and R64, a routing-based proxy, finding that both improve test-time selection and control, with routing preserving much of J64’s predictive gain.

  • Problem

    Reasoning traces record emitted text rather than the full process state, limiting direct visibility into active constraints, inference effort, strain, and unproductive branches.

  • Method

    The paper learns J64 as a compact semantic frame from reasoning states and reconstructs it as R64 from native MoE routing statistics for test-time decisions.

  • Results

    J64 and R64 improve completed-rollout selection, routing-weighted voting beats majority voting in seven of eight settings, and R64 preserves 95–100% of J64’s predictive gain on gpt-oss-20b.

  • Takeaways & Limitations

    J64 makes latent reasoning state readable, while R64 turns native routing into low-overhead telemetry that supports actionable test-time selection and stop-and-resample control.

  • Takeaways & Limitations

    The analysis centers on competition mathematics and gpt-oss-20b, with online control evaluated mainly by causally masked replay and broader domains left for future work.

Abstract

from arXiv · show

What a reasoning model writes is only a partial record of the process that produces it. We introduce a two-level internal readout for mixture-of-experts reasoning. We first distill vocabulary-scale J-space into J64, a 64-axis semantic frame learned from the model's own reasoning states. J64 reveals readable process state that the emitted trace does not show: it separates inference effort from problem-induced strain. It also adds 0.096 to 0.135 held-out AUC over a baseline that reads the same rollout as token occupancy and aggregates it in exactly the same way. We then reconstruct J64 from native expert-routing statistics. The result is R64, a low-overhead proxy: its median per-axis correlation with J64 is 0.69 to 0.86 across three models and two families, and on gpt-oss-20b it preserves 95 to 100% of J64's predictive gain. The readout supports test-time decisions at two temporal resolutions. Over completed candidate sets, J64 and R64 improve single-branch selection, and R64-weighted voting improves plain majority voting in seven of eight settings. During generation, rolling readout windows drive a cumulative stop-and-resample policy whose operating point is fixed on training questions alone. J64 improves accuracy by 1.1 to 5.9 points over a sibling-permuted control, and the routing-only R64 proxy retains 0.9 to 3.2 of those points. Finally, router edits aimed at the mechanism J64 names induce the predicted reasoning behaviors and shift a diagnosed stall from numerical guessing toward exact symbolic execution. Together, J64 makes latent process state readable, while routing makes it deployable and actionable.

1 Introduction

The paper introduces J64 as a compact, interpretable readout of latent reasoning state beyond emitted traces, then reconstructs it as R64 from native MoE routing. These readouts support rollout selection, weighted voting, online stop-and-resample decisions, and mechanism-targeted router edits.

  • Motivation: Reasoning traces expose emitted text but not active constraints, alternatives, effort versus difficulty, or whether a branch is becoming unproductive.Matched rollouts can emit identical local spans while differing in internal state and eventual correctness.
  • J64: J64 compresses vocabulary-scale J-space into 64 semantic families derived from the model’s own reasoning states without outcome, effort, or difficulty labels.Its axes are named only after construction and map hidden states to a readable reasoning dashboard.
  • J64: 0.096–0.135 held-out AUC beyond a same-aggregation token-occupancy baseline shows that J64 separates inference posture from problem-induced strain.J64 is a coordinate system for latent reasoning state, not a correctness classifier or predicted-token list.
  • R64: 0.69–0.86 median per-axis correlation across three models and two families, with 95–100% of J64’s predictive gain preserved on gpt-oss-20b, establishes R64 as a low-overhead routing proxy.R64 reconstructs the same 64 semantic coordinates from native expert assignments and gate weights.
  • Test-time use: Seven of eight settings improve when voting is R64-weighted rather than plain majority voting, while telemetry also supports rollout selection, online stop-and-resample, and mechanism-targeted router edits.The edits change reasoning in the direction predicted by the named readout mechanism.

2 Related Work

Prior work shows that internal activations can reveal information absent from model outputs, while test-time selection and MoE routing typically rely on text, output confidence, or systems-oriented routing signals. This work instead uses internal reasoning state and native routing statistics as semantic telemetry for selection, stopping, and resampling.

  • Reading internal states: Vocabulary-aligned lenses and linear probes expose information that model outputs do not directly show, including silent intermediate reasoning in J-space.The paper uses this readout operationally rather than testing J-space’s global-workspace properties.
  • Test-time compute and selection: Internal-state or routing-proxy selection differs from repeated sampling, majority voting, process reward models, lengthened reasoning, and text-only self-correction by reading telemetry beyond the trace.The same telemetry also supports stopping and resampling.
  • MoE routing: MoE routing is usually studied for capacity, load balancing, and expert specialization, whereas behavioral control commonly writes directions into the residual stream.DeepConf is the closest online comparison: it filters or early-stops traces using token-level output confidence without training; this work treats routing as a semantic sensor after one-time calibration.

3 Method and Protocol

The method defines J64 as a 64-axis, vocabulary-aligned readout of hidden states and learns R64 as a label-free proxy from native MoE routing. It evaluates both at trajectory and rolling-window resolutions under question-level, frozen-transfer statistical controls.

  • J64 readout: J64 represents each trajectory by the mean of 64 concept readings from a model-specific, single-layer hidden-state lens.The frame uses a pseudo-inverse because its axes are non-orthogonal and includes reasoning concepts alongside text-production markers.
  • R64 proxy: R64 maps native routing spectra to the 64 J64 axes without outcome labels.The routing spectrum aggregates per-expert gate weights and one gate-entropy feature per layer, yielding 99 features for gpt-oss-20b and 387 for two 128-expert models.
  • R64 proxy: R64 avoids a second forward pass by reading expert assignments and gate weights already emitted during generation, then applying aggregation and a light regression head.J64 requires replaying the trajectory because the generation stack does not emit activations.
  • Temporal resolutions: Routing is analyzed as trajectory-aggregated readouts in RQ2–RQ3 and rolling-window readouts in RQ4, with separate parameters and a distinct prefix-failure head.R64 predicts the readout from the full routing spectrum, whereas RQ5 intervention targets come from J64 geometry or outcome-linked usage.
  • Statistical protocol: The protocol groups models and confidence intervals by question and uses frozen cross-benchmark transfer, training each fitted component on one benchmark before testing the other three.The J64 frame uses no labels; supervision differs across the selection and online-control heads.

4 Building an Interpretable Routing Readout

J64 provides a compact, interpretable readout of latent reasoning state beyond the emitted trace, while R64 reconstructs that state from native expert routing. Routing preserves nearly all of J64’s predictive content and localizes named process states to expert modules.

  • J64 beyond the trace: 99.6% of above-threshold axis readings have no nearby token-family member, showing J64 is not a direct lexical echo.For 50 of 64 axes, the figure is exactly 100%; family words appear at most 0.7 times per thousand tokens.
  • Interpretable process coordinates: J64 organizes readable process distinctions through posture, which contrasts case-splitting with arithmetic-core activity, and strain, which tracks rollout correctness.The posture coordinate separates three effort settings; strain correlates negatively with correctness at −0.32, −0.28, and −0.35 across effort levels.
  • J64 beyond the trace: J64 adds outcome-relevant process information beyond a matched text channel that summarizes emitted-token occupancy in the same order-free way.The comparison changes only the signal source, isolating what is read from how the rollout is aggregated.
  • Routing reconstruction: 0.692–0.864 median per-axis held-out reconstruction shows routing usage can recover J64 across eight model–effort settings.The two Qwen variants reconstruct at 0.864 and 0.833, while single-token routing still reaches r = 0.42–0.48.
  • Routing reconstruction: +0.128∗/+0.129∗/+0.093∗ outcome AUC at Low/Medium/High means R64 retains 95–100% of J64’s increment over matched text.The routing proxy needs no additional forward pass, and the lens is queried only once at calibration.
  • Routing localization: Routing modules track named J64 states token by token, including drift at r = +0.79 and symbolic activity at r = +0.59 within a ±72-token window.Peak locations align with the hypothesized states: drift at a paragraph break and symbolic activity inside a mathematical expression.

5 Acting on the Readout at Test Time

At test time, J64 and its routing-only proxy R64 improve branch selection, vote aggregation, and cumulative stop-and-resample decisions under frozen or question-held-out evaluation. Router-logit edits aimed at J64-diagnosed mechanisms produce the predicted reasoning changes, linking interpretable state to actionable routing control.

  • Offline selection and aggregation: J64 and R64 improve single-branch selection over random choice across every model group, with J64 gaining +5.0 and +4.9 points at the two gpt-oss scales.R64 remains positive in all four model groups, with gains of +2.6, +2.5, +8.4 and +3.8 points.
  • Offline selection and aggregation: Routing-weighted voting improves plain majority voting in seven of eight settings and in all four model-group averages.The routing-weighted gains are +0.4/+1.6/+0.3/+2.3.
  • Online stop-and-resample: A CUSUM controller uses 256-token windows of J64, R64 or DeepConf failure-risk scores to stop unproductive attempts and restart from fresh samples.The operating point is nested and frozen using training questions, with held-out questions used for final accuracy and realized cost.
  • Online stop-and-resample: J64 beats the sibling-permuted control at every target cost, gaining +1.3∗ to +5.9∗ points on gpt-oss-20b High and +1.1∗ to +1.7∗ on Qwen.R64 retains +2.2 to +3.2∗ points on High and +0.9 to 3.2 of those points overall as the routing proxy.
  • Mechanistic intervention: Targeted router-logit edits induce the predicted reasoning behavior: amplifying experts advancing a diagnosed case-splitting bundle causes sustained “Case n” enumeration, reaching 100 headers in one run versus none in 160 sham runs.Suppressing a diagnosed stall links an overused routing group to a J64 state that maintains problem requirements instead of executing the requested operation.

6 Limitations and Conclusion · A Supplementary Analyses

The study makes process state readable through J64, reconstructs it from native routing with R64, and uses both for selection, online control, and causal case studies. Its evidence is centered on competition mathematics and gpt-oss-20b, with replication on two additional models and supplementary analyses documenting the constructions and protocols.

  • 6 Limitations and Conclusion: The analysis centers on competition mathematics and gpt-oss-20b.This defines the primary scope of the reported analysis.
  • 6 Limitations and Conclusion: Reconstruction and selection replicate on gpt-oss-120b and Qwen3-30B-A3B, but each model’s frame is built independently.The independently built frames are not aligned with one another.
  • 6 Limitations and Conclusion: Online control is evaluated mainly by causally masked replay, and the interventions remain small.These qualifications limit how broadly the online-control and intervention evidence should be interpreted.
  • 6 Limitations and Conclusion: The readout tracks process state and, before an answer is committed, does not predict the eventual outcome.The supplied passage also notes that activity on reflective vocabulary is largely [text truncated].
  • 6 Limitations and Conclusion: J64 makes process state readable beyond the emitted trace, while R64 reconstructs it from native routing without activation replay.The paper frames J64 as semantic readout and routing as the deployment path.
  • 6 Limitations and Conclusion: The same telemetry supports completed-rollout selection and online stop-and-resample control.These are the two stated temporal resolutions for test-time use.
  • 6 Limitations and Conclusion: Edits aimed at the named mechanism provide causal case studies, making the signal actionable through test-time policies.The conclusion distinguishes readable semantics, deployable routing, and actionable policies.
  • A Supplementary Analyses: The supplementary analyses are grouped by research question and follow the order of the main text.Appendix B specifies every construction, protocol, and constant used by the main text.

A.1 RQ1: The Readout and Where Its Increment Sits

The readout’s correctness signal is compact, while effort classification and best-of-N selection require the full 64-axis dashboard. Frame-disjoint validation preserves the principal effects, and a capacity-matched PCA64 basis spans nearly the same subspace and performs similarly.

  • Compactness: 85–95% of the correctness increment transfers through an eight-axis panel, but effort classification and best-of-N selection require the full dashboard.An eight-axis frame loses 9pp on selection.
  • Frame-disjoint stability: 0.938 AUC effort classification holds on 96 frame-disjoint questions versus 0.935 on the full pool, while difficulty AUC remains 0.694/0.794/0.668.Strain–difficulty correlations are 0.434/0.500/0.445 on the disjoint questions versus 0.410/0.460/0.405 on the full pool.
  • Capacity-matched PCA control: 0.986–0.999 top-five canonical correlations show that J64 and PC64 span nearly the same subspace across every effort.Selectors trained on either basis score the same rollouts alike, with r = 0.74–0.88.
  • Capacity-matched PCA control: −3.3 [−7.8, +1.1], −2.2 [−6.7, +2.5], and −1.7 [−4.0, +0.3] points are the question-paired PC64–J64 selection differences at Low/Medium/High.Concatenating the two readouts improves neither arm.

A.2 Replication at 120b Scale

At 120b scale, standalone J64 replicates and strengthens the readout result, while stacking text with J64 adds nothing over text alone. The independently extracted frame is not axis-aligned with the 20b frame, and truncation decreases substantially across effort settings.

  • Standalone readout: J64 alone reaches 0.794/0.761/0.865 versus 0.585/0.613/0.722 for the matched full-trace baseline across three effort settings.The experiment uses 7,680 rollouts per effort setting.
  • Stacked estimator: +0.004/+0.001/−0.001 is the stacked text+J64 increment over text, with confidence intervals crossing zero.At 120b, regularized occupancy-feature fitting is dominated by the text block, so the single-arm duel is reported.
  • Scale and failure mode: 41% truncation at High falls to 18%, and to zero at Low and Medium.The 120b frame is extracted independently at that model’s own layer, so scale comparisons use frame-wide statistics rather than axis-by-axis alignment.

A.3 RQ2: The Routing Proxy and What It Reads

R64 reconstructs J64 from native routing with strong held-out alignment and preserves its outcome-relevant readout behavior. The routing signal primarily tracks process state and trajectory quality, while interpretable axes and modules clarify what it reads and why the full spectrum remains preferable for selection.

  • Reconstruction quality: 0.833 median Pearson r reconstructs J64 for Qwen-Instruct, with 58/64 axes above 0.7 versus −0.019 for the row-permuted control.The identical protocol reproduces Thinking at 0.864, 59/64 axes above 0.7, and −0.022 for the permuted control.
  • Reconstruction quality: R64 inherits J64’s label structure: its High-effort outcome-AUC increment crosses zero under the finished-answer convention, while Low and Medium remain positive.This matching pattern is presented as evidence that the proxy tracks where J64 is informative and where it is not.
  • Bottleneck and deployment: −0.1 points is the average question-paired direct-minus-R64 gap across eight settings, with the direct arm leading in three and Qwen-Instruct favoring R64 by −3.9 [−7.5, −0.3].The comparison uses frozen cross-benchmark transfer, one shared regularizer, and finite training sets.
  • What the axes mean: Axis meaning is directional: semantically coherent bipolar neighbor families reveal state markers, including drift off the English manifold, echo hazards, and systematic correctness-related sign structure.Keep/advance/continuation families load positive, whereas alternatives/meta-planning/complexity families load negative; most axes have garbled negative poles and are unidirectional features.
  • What routing reads: Routing predicts trajectory correctness but not answer-string commitment: answer prediction is at chance at 0.493, whereas correctness prediction is where routing matches the lens.This establishes routing as a readout of process state and completion dynamics rather than exact answer content.

A.4 RQ3: Branch Selection and Vote Weighting

On GPQA, question-held-out, question-residualized selection evaluates single-branch prediction unconditionally across all question pools. J64’s advantage depends on setting failure modes and remains bounded by the fraction of pools containing a correct branch, while weighted voting adds little on the rescue slice.

  • Failure modes: At 120b, trace length is harmful in Low and Medium because failures are pure capability errors; at High, 18% truncation makes length the strongest single-trace arm.The reported length effects are −13.8 and −8.3 points in Low and Medium, respectively.
  • Consensus and voting: On the oracle-conditioned rescue slice, weighted-vote variants do not improve over random single-branch selection, while DeepConf-weighted voting falls 4.3 points below it.This indicates the internal score contributes information beyond simply counting votes.
  • Protocol: Table 5 evaluates best-of-64 single-branch selection on 198 graduate-level science questions using question-held-out folds and question-residualized features.Every question–setting unit is included, with gains reported against the same-column random-pick reference.
  • Headroom: Only 45–75% of pools contain any correct branch, limiting the oracle ceiling; at High effort, J64 captures about six-tenths of the available oracle-minus-random gap.At Low effort with N=16, the oracle reaches only 0.56–0.76.
  • Pool composition: At N=64, pool composition varies by effort, with all-correct / all-wrong / mixed counts of 0–1 / 4–9 / 21–26 at Low, 2–7 / 0–2 / 23–26 at Medium, and 3–10 / 1–3 / 18–25 at High.Recomputing arms on mixed pools leaves their ordering unchanged across efforts.

A.5 RQ4: Online Compute Allocation

Table 3 compares J64 and R64 against the sibling-permuted control, the basis for the online claim. R64 trails J64 on gpt-oss-20b High, is equivalent on Qwen, and leads at the cheapest target.

  • Table 3 uses the sibling-permuted control as the comparison underlying the online claim.The table compares both readout forms against that control.
  • +1.4∗to +2.7∗points: R64 trails J64 on gpt-oss-20b High.The margin occurs at the higher budgets of one model group.
  • 0.9∗ at 1.2×: R64 leads J64 at the cheapest target, while the two readouts are equivalent on Qwen.The ordering therefore inverts at the cheapest target.

A.6 RQ5: Mechanism-Matched Routing Interventions

Mechanism-matched routing interventions selected experts either from J64 directions or routing-usage statistics, producing targeted reasoning changes in gpt-oss-20b. The strongest behavioral shift changed a diagnosed numerical stall into exact symbolic execution, while intervention effects depended on narrow expert-level doses.

  • Target selection: Experts were selected either geometrically from a J64 bundle direction or statistically from routing usage, rather than by R64 reconstruction coefficients.The two rules measure different quantities: an expert can rank 80th of 96 by reconstruction weight yet remain a core member of the usage-based target group.
  • Scope and limitation: Expert-level doses of 0.3–0.8 worked, whereas larger doses returned performance to baseline.The intervention arena was gpt-oss-20b at Low effort on competition mathematics, and one probing axis was interpreted as a state marker rather than literally.
  • Intervention outcomes: 0.411 accuracy followed promotion of outcome-positive experts at dose 0.3, versus 0.380 for no-op and 0.359 for equal-dose sham.Suppressing the negative group at dose 0.8 reached 0.406; among single-direction treatment arms, treatment–sham separation was +3.6 to +5.2 points.
  • Behavioral effects: −45.9 tokens [−75.7, −17.3]∗ under suppression contrasted with +47.6 [+13.5, +82.9… under the comparison condition.The targeted amplification arm produced named case structure, whereas the equal-dose sham repeated a single sentence despite equal accuracy loss.
  • Mechanism diagnosis: Suppressing diagnosed experts changed a tetrahedron solution from decimal guessing toward exact symbolic execution, including V = 160/3 and r = 20/21.The unmodified trajectory guessed a radical and answer 12, while the intervention preserved exact symbolic form throughout.

B Reproducibility: Constructions and Protocols

This section defines every construction used in the main text and records the implementation values that produced the reported artifacts.

  • The section specifies every construction used by the main text.
  • The reported values are taken from the code used to produce the artifacts.
  • The constructions and values together establish the protocols underlying the reported artifacts.

B.1 The J64 Frame and the Two Needles … B.6 Glossary

The paper defines a label-free, model-specific J64 semantic frame, compares it with a matched token channel, and reconstructs it from native MoE routing. It then applies these readouts to selection, voting, and prefix-based stopping under controlled evaluation protocols.

  • B.1 The J64 Frame and the Two Needles: The frame uses 24 construction questions, while headline evaluations use all 120 and additional construction-disjoint controls use the remaining 96.Axes 30 and 1 are respectively the problem-perception and constraint-requirement axes, with axis sets chosen on one random half and confirmed on the other 60 problems.
  • B.1 The J64 Frame and the Two Needles: J64 is built without outcome, effort, or difficulty labels using a per-source-layer Jacobian lens and model-specific readout layers.The lens reads layer 20 of 24 for gpt-oss-20b, layer 23 of 36 for gpt-oss-120b, and layer 34 of 48 for Qwen3-30B-A3B.
  • B.2 Labels, the Matched Text Channel and the RQ1 Estimator: RQ1 defines success as a correct answer delivered before the generation limit, whereas completed-pool selection scores a branch by extracted answer alone.The matched text channel keeps the 3,000 most frequent training-fold tokens, applies log(1+x), ℓ2-normalization, and fold-specific standardization.
  • B.3 R64: Routing Features and the Reconstruction Map: Routing features comprise 99 statistics for gpt-oss-20b and 387 for gpt-oss-120b and Qwen3-30B-A3B, combining normalized expert gate usage with three layerwise entropies.Routing is recorded during generation, while J64 readings require replaying finished trajectories under teacher forcing with a residual-stream hook.
  • B.4 RQ3: Selection, Transfer and Voting Protocols: Selection uses 64-rollout pools, frozen logistic selectors, argmax single-branch picks, and probability-weighted voting over terminated branches with parseable answers.The main protocol fixes inverse regularization at C=3 and reports macro-averages over 12 source→target pairs.
  • B.4 RQ3: Selection, Transfer and Voting Protocols: The construction-disjoint transfer check removes the 24 frame-construction questions from selector fitting and evaluation, leaving 24 questions per target set.On this subset, every arm loses 2–4 points of absolute accuracy, including the random-pick reference, while the unrestricted rerun reproduces the original cells.
  • B.5 RQ4: Prefix Score, Controller and Baselines: Prefix control uses 256-token windows with stride 256, a gradient-boosted failure predictor, and CUSUM stopping with thresholds τ ∈{0.25, 0.5, 1, 2, 4} and restart budgets K ∈{2, 3, 4, 6}.permSib is the attribution baseline because it preserves marginal, autocorrelation, and question-level information while destroying branch-level information.
  • B.6 Glossary: The glossary defines a vocabulary direction, semantic frame, state marker, and echo hazard for interpreting J64 readings and avoiding literal readings of digit axes.A semantic frame is A = [a1 ··· a64] ∈R^d×64, while state markers reflect manifold position rather than content words.
Loading 2608.17638v1…