PaperlayerView exact revision on arXiv

Source-linked AI summary

Deeper is Not Always Better: Mitigating the Alignment Tax via Confident Layer Decoding

Xuanming Zhang, Sining Zhoubian, Yuxuan Chen, Tianyi Tang, An Yang, Sean Du, Chujie Zheng, Fei Huang, Dayiheng Liu, Gao Huang, Jingren Zhou

arXiv:2606.21906v1cs.CL

TL;DR

Final-layer representations are not always the most reliable source for reasoning-sensitive next-token prediction, so this paper introduces training-free Confident Decoding to dynamically select a near-final layer through entropy-guided backward search. Across dense and Mixture-of-Experts models, it consistently improves over greedy and strong contrastive baselines, with negligible memory overhead and less than 2% additional latency.

  • Problem

    Whether final-layer representations are always the most reliable for reasoning-sensitive next-token prediction is uncertain, as intermediate layers can encode task-relevant semantics while later layers may perturb them.

    Source: §1 · p. 2

  • Method

    Confident Decoding is a training-free strategy that dynamically selects near-final logits through entropy-guided conservative backward search.

    Source: §1 · p. 2

  • Results

    Across dense and Mixture-of-Experts architectures, Confident Decoding consistently improves over standard greedy decoding and strong contrastive baselines, with negligible memory overhead and less than 2% additional latency.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Final-layer representations should not be assumed to be the most reliable source for reasoning-sensitive next-token prediction.

    Source: §1 · p. 2

    The approach depends on alignment between the unembedding matrix WU and intermediate residual states, and shallow-layer representations may still have vocabulary mismatch.

    Source: §8 · p. 15

Abstract

from arXiv · show

Autoregressive generation in large language models (LLMs) conventionally decodes from the final layer, assuming that deeper representations yield more reliable next-token predictions. We revisit this assumption by revealing a recurring Guess-Refine-Perturb dynamic: early layers form coarse guesses, intermediate layers refine reasoning-relevant semantics, and final layers can perturb these refined predictions toward generic or alignment-preferred tokens. We introduce Confident Decoding, a training-free decoding strategy that dynamically selects the most reliable near-final layer through entropy-guided conservative backward search. We further provide a theoretical formulation of layer selection as an optimal stopping problem, showing that under bounded projection noise and dominant late-stage alignment perturbation, our search rule filters perturbation while bounding the loss relative to the oracle refinement layer. Experiments across dense and Mixture-of-Experts LLMs demonstrate consistent gains on challenging reasoning benchmarks, including GPQA-Diamond, Omni-MATH, and HLE, with zero memory overhead and less than 2% latency increase. These results suggest dynamically bypassing final-layer perturbations can unlock stronger reasoning behavior from aligned LLMs.

1 Introduction

The paper argues that final-layer predictions can drift from reasoning-relevant intermediate representations, and introduces Confident Decoding to select a more reliable near-final layer. It reports consistent benchmark gains with minimal inference overhead.

Motivation

  • The authors identify a Guess–Refine–Perturb pattern: early layers form coarse guesses, intermediate layers refine them, and final layers can shift predictions away from reasoning-relevant states.The late-stage shift occurs for a subset of tokens.Source: §1 · p. 2
  • The authors hypothesize that post-training alignment can bias final-layer distributions toward frequent, safe, or generic continuations, conflicting with intermediate reasoning trajectories on complex tasks.They call this tension a planning–pragmatics tradeoff; alignment behavior may still serve as a useful guardrail for conversational or safety-oriented prompts.Source: §1 · p. 2

Related work

  • Existing contrastive, logits-evolution, and early-exit methods do not directly test whether the final layer should always emit the next-token distribution.The paper says prior methods retain the final layer as their main semantic anchor or primarily target inference efficiency.Source: §1 · p. 2

Method

  • Confident Decoding preserves the full forward pass and selects near-final logits at the first local entropy trough found by conservative backward search.The training-free, drop-in method uses lower token entropy as a signal of a sharper predictive distribution before potential late-layer perturbation.Source: §1 · p. 2

Evaluation overview

  • Across dense and Mixture-of-Experts models, Confident Decoding improves over greedy decoding and strong contrastive baselines, with negligible memory overhead and less than 2% additional latency.The evaluation spans reasoning, mathematics, long-context understanding, coding, and safety benchmarks.Source: §1 · p. 2
  • Larger and stronger models can show more pronounced late-layer perturbations, making dynamic layer selection increasingly beneficial as model capability and task difficulty grow.The paper also presents theoretical and empirical evidence that the method filters late-layer perturbations, preserves refined reasoning signals, and improves performance across diverse benchmarks.Source: §1 · pp. 2–3

2 Preliminaries

The preliminaries characterize a three-phase progression in layer representations and motivate token-adaptive entropy-based layer selection over static exits. They identify the Entropy Valley as a local confidence boundary near the end of refinement.

2.1 Layer-wise Dynamics

  • Relative Contribution Norm measures layer write intensity, while IO-CosSim measures directional fidelity and whether a layer preserves or rotates the semantic trajectory (Skean et al., 2025; Mikolov et al., 2013).A Norm Ratio near or below 1 alone cannot distinguish constructive reinforcement from disruptive rewriting.Source: §2.1 · pp. 3–4
  • In Qwen3.5-35B-A3B, Phase I has Norm Ratio(1) ≈1.6 and IO-CosSim(1) ≈0.67; Phase II stabilizes at 0.23–0.57 and 0.91–0.97, respectively.The initial phase forms a latent representation under high uncertainty; the intermediate phase makes incremental, directionally faithful updates.Source: §2.1 · p. 4
  • At layer 40, Norm Ratio reaches 2–3× the Phase II level and IO-CosSim falls to ≈0.69, marking a directionally misaligned update that perturbs refined semantics.The late-layer write remains below 1, so it does not enter the Phase I overwriting regime.Source: §2.1 · p. 4

2.2 Motivation: Emerging Entropy Valley in Intermediate Layer

  • As static early-exit probability increases, accuracy declines sharply because uniform truncation can interrupt computation needed for hard tokens.The passage motivates a token-adaptive indicator to select depth for each hidden state.Source: §2.2 · p. 4
  • On GPQA-Diamond, dynamic valley selection outperforms last-layer decoding and static early exits, while selecting neighboring valley layers degrades performance.Figure 3 supports the valley as a token-dependent decoding depth near the Phase II/III boundary.Source: Figure 3 · p. 5, §2.2 · p. 5
  • The Entropy Valley is the local entropy minimum found by scanning backward from the final layer, balancing final-head mapping reliability against intermediate confidence before possible late oscillations.The original unembedding head is optimized for the final-layer latent space, so mapping error increases farther from the final layer.Source: §2.2 · pp. 4–5

3 Theoretical Grounding

The analysis describes an intermediate-depth representational optimum: entropy may rise when final layers perturb reasoning predictions, while conservative backward search is formulated to filter those perturbations and bound semantic-precision loss.

3.1 Information-Theoretic View of Layer Dynamics

  • On Qwen3.5-35B-A3B, 16.2% of tokens gained +0.37 nats of entropy at the final layer, while 83.8% gained a further −2.52-nat entropy drop through refinement.The minimum-entropy layer is V*=39, after refinement accumulates approximately −10.8 nats across 202,935 tokens; the perturbation is per-token rather than a fixed layer region.Source: §3.1 · p. 6

3.2 Modeling Alignment: Tax vs. Guardrail

  • Final-layer alignment acts as a guardrail when task and generic safe distributions agree, but as a tax when complex-reasoning distributions conflict, steering representations away from the reasoning subspace.The paper attributes late-stage alignment behavior to low-rank steering vectors predominantly activated in final layers (Zou et al., 2023).Source: §3.2 · pp. 6–7

3.3 Minimax Optimality of Conservative Backward Search

  • Under bounded projection noise and dominant conflicting late perturbations, backward stopping selects within [Vonset, V*], filtering alignment perturbations while bounding semantic-precision loss.The theorem defines the selected layer as the latest l < L where observed entropy does not decrease from l−1 to l.Source: §3.3 · p. 7
  • Strong refinement selects the oracle layer V*, while weak-signal oscillations may stop earlier with a theoretically negligible semantic deficit; the resulting performance is a lower-bound approximation to greedy decoding.For strong integration, the stopping rule is oracle-optimal; for weak integration, the semantic loss is bounded by the diminished refinement gradient.Source: §3.3 · p. 7

4 Methodology

Confident Decoding is a training-free inference-time method that selects logits from a dynamically chosen near-final layer using predictive entropy. It keeps the full forward pass and sampling pipeline unchanged, forwarding only the selected layer’s logits to the sampler.

4 Methodology

  • At each decoding step, Confident Decoding forwards logits from the near-final layer with the lowest predictive entropy instead of always using the final layer.The full model forward pass, weights, KV cache, and downstream sampling remain unchanged; only the layer whose logits reach the sampler changes.Source: §4 · pp. 7–8

4.1 Confident Decoding

  • Within a configurable near-final candidate set, the default rule scans backward and selects the first local entropy valley, freezing the choice at the first non-improvement.The candidate-set size or fixed scan window controls backtracking, with larger windows increasing computational cost.Source: §4.1 · p. 8
  • Fallback probability p mixes entropy-valley selection with standard final-layer decoding: p = 0 recovers standard decoding, while p = 1 always selects the entropy valley.The stochastic fallback occurs with probability 1 − p.Source: §4.1 · p. 8

4.2 Systems Implementation

  • The method executes all transformer blocks and leaves the KV cache, attention kernels, prefix caching, and continuous-batching scheduler unchanged.Only the layer supplying sampler logits changes, allowing composition with existing speculative-decoding, multimodal, and tensor-parallel infrastructure.Source: §4.2 · p. 9
  • The eager wrapper applies final normalization to captured candidate states and aligns sliced candidate logits with sampler inputs across recorded graph shapes.Keeping stateful logic outside the compiled region and using consume-once indices prevents replay state from leaking across calls.Source: §4.2 · p. 9
  • O(MBd|V|) is the overall extra computation cost, while additional memory is negligible compared with the KV cache.The overhead comprises batched candidate unembedding, entropy evaluation, and a K-step trough scan; M controls worst-case overhead and K ≤ M bounds scan cost.Source: §4.2 · p. 9

5 Experiments

Across dense and MoE models, Confident Decoding yields consistent reasoning gains. Instruct/base comparisons show larger average gains for the instruction-tuned model, while context-heavy and safety-task performance remains stable or improves.

5.1 Experimental Setup

  • Experiments use Qwen3.5-35B-A3B as the primary backbone, with a lookback window K = 10 and deterministic valley selection; evaluation spans six reasoning, math, context, coding, safety, and writing benchmarks.Sampling temperature is 0, matching the greedy baseline.Source: §5.1 · pp. 9–10

5.2 Main Results

  • Confident Decoding improves reasoning across dense and MoE models, including +10.1% absolute on LiveCodeBench-v6 for Qwen3.5-27B, +6.5% on GPQA-D for Qwen-35B, and +2.4% on HLE.The reported gains also extend to larger models, including gpt-oss-120B and Qwen3.5-122B.Source: Table 1 · p. 10, §5.2 · pp. 10–11
  • WritingBench gains are only +0.1% to +0.5%; LongBench-v2 has ∆≈0, while Air-Bench gains +5.0% for both models, preserving safety and context-task performance.The paper attributes LongBench-v2 stability to strong contextual constraints and reports that Air-Bench performance does not degrade when final layers are bypassed.Source: §5.2 · p. 11, §5.3 · p. 12

5.3 Instruct vs. Base Model: Validating the Alignment Tax

  • Confident Decoding yields average absolute gains of +2.8% for the Instruct model versus +1.7% for the Base model, supporting a larger alignment tax after instruction tuning.On HLE, greedy decoding scores 7.1% for Instruct versus 8.0% for Base, while Confident Decoding raises Instruct to 9.5%.Source: Table 2 · p. 11, §5.3 · p. 11
  • Entropy valleys occur for 12.8% of Instruct versus 10.4% of Base tokens; substitution rates are 2.60% versus 2.36%, with substitution-position ∆H of 3.48×10−2 versus 3.34×10−2.About 21% of identified valleys change the decoded token; the remainder alter entropy without changing the argmax.Source: §5.3 · p. 12

6 Discussions

The discussion shows that Confident Decoding intervenes selectively where entropy signals final-layer perturbation, with larger gains on harder reasoning tasks and positive average gains across model backbones. It reports sparse projection overhead, zero additional KV-cache memory, and less than 2% end-to-end latency increase.

6.1 Entropy Valley Dynamics

  • The backward scan selects an entropy valley for 11.5% of tokens, and 21.4% of those scans change the argmax, yielding a 2.47% overall substitution rate.Substitution is effectively zero for H < 0.01 and rises to 14.8% above H = 0.10; valley and substitution rates peak mid-sequence, with the instruct-tuned model reaching 17.4% and 3.94%, respectively, at the 60–70% mark.Source: §6.1 · p. 12

6.2 Task Complexity vs. Algorithm Efficacy

  • On Level 4 Omni-MATH, gpt-oss-20b improves by +22.4 points, while Confident Decoding is −0.4% on near-perfect Level 1 MATH.The performance delta grows with task complexity; the source reports only 1.1% accuracy for gpt-oss-20b on the hardest Omni-MATH problems before the +22.4-point improvement.Source: §6.2 · pp. 12–13, Table 3 · p. 13

6.3 Architectural Robustness

  • Across six backbones, Confident Decoding yields positive average gains, with the largest gains on reasoning-intensive benchmarks and small gains on context-saturated tasks.The four MoE backbones gain +2.8 (Qwen3.5-35B-A3B), +1.7 (gpt-oss-120B), +1.5 (gpt-oss-20b), and +1.1 (Qwen3.5-122B-A10B); the entropy measure smooths routing volatility and the backward search filters routing noise.Source: §6.3 · pp. 13–14

6.4 Computational Overhead and Real-World Viability

  • The backward scan averages 0.116 iterations per token, while deployment adds zero KV-cache memory and strictly < 2% per-token wall-clock latency.The empirical FLOPs increase is < 1%; 88.5% of tokens require zero additional projections, despite a theoretical worst-case 74.6% FLOPs increase at K = 10.Source: §6.4 · p. 14, Table 5 · p. 14

7 Related Work

Prior work shows that intermediate layers can preserve useful predictions and reasoning signals that later layers may perturb. Confident Decoding builds on this evidence by selecting a layer without relying on final-layer logits and treating internal stopping as an efficacy strategy.

Layer-wise Dynamics and Mechanistic Interpretability

  • Intermediate-layer evidence challenges monotonic depth assumptions and motivates the Guess-Refine-Perturbation formalization.Prior studies find predictions crystallize before the output head (Belrose et al., 2023), factual knowledge localized to mid-layer MLPs (Meng et al., 2022), and intermediate representations stronger than final-layer ones (Skean et al., 2025); intermediate decoding can also reveal reasoning paths suppressed by final-layer outputs (Wang and Zhou, 2024).Source: §7 · pp. 14–15

Contrastive and Adaptive Decoding

  • Unlike contrastive methods that anchor on final-layer logits, Confident Decoding uses an entropy valley to avoid corrupted terminal distributions.Prior approaches include expert-amateur contrastive decoding (Li et al., 2023b), DoLa’s final-versus-early-layer contrasts (Chuang et al., 2023), and adaptive or entropy-guided refinements (Das et al., 2025).Source: §7 · p. 15

Test-Time Computation and Optimal Stopping

8 Conclusion · Appendix

The paper presents Confident Decoding as a training-free way to bypass late-layer alignment perturbations while limiting semantic loss, and identifies vocabulary mismatch and training-time alignment as unresolved limitations. It proposes decoupling reasoning from alignment, testing whether the dynamics persist in multimodal models, and using layer-wise entropy to improve reinforcement-learning rewards.

8 Conclusion

  • Confident Decoding improves frontier-reasoning performance across dense and MoE architectures with negligible, bounded computational overhead.The conclusion cites GPQA-Diamond and Omni-MATH as examples.Source: §8 · p. 15
  • The method uses a risk-averse optimal-stopping backward scan to locate a monotonic entropy valley, isolate alignment-induced perturbations, and bound semantic loss.It is training-free and addresses the tradeoff between planning and pragmatics.Source: §8 · p. 15
  • Confident Decoding depends on alignment between the unembedding matrix WU and intermediate residual states, while shallow representations may still have vocabulary mismatch.The theoretical framework bounds projection noise but does not eliminate this limitation.Source: §8 · p. 15
  • The approach mitigates alignment-tax symptoms during decoding rather than resolving their root cause during training.This limitation qualifies its role as an inference-time intervention.Source: §8 · pp. 15–16

Future Directions

  • Future work could decouple pre-trained reasoning from post-training pragmatics by applying alignment penalties to designated routing heads instead of the core residual stream.The paper frames this as a direction for developing natively robust reasoning models.Source: §8 · p. 16
  • Future work could test whether Guess-Refine-Perturbation dynamics persist in multimodal foundation models.The paper identifies this as a frontier for mechanistic exploration.Source: §8 · p. 16
  • Layer-wise entropy metrics could inform more geometrically precise reward functions for reinforcement learning.The proposed direction aims to support natively robust reasoning models.Source: §8 · p. 16

A Hyperparameters and Configurations · B Three-Phase Structure on More Models

The section reports that deterministic valley selection with greedy decoding performs best in the p-and-temperature ablation, and that the three-phase pattern also appears in a smaller model. It specifies the decoding defaults and benchmark configuration conventions used in the experiments.

A Hyperparameters and Configurations

  • Main experiments generally use K=10 and p=1.0: the search window covers most entropy valleys, while deterministic selection avoids mixing in final-layer predictions.Across six benchmarks and all evaluated model scales, valleys rarely lie more than five layers below the final layer.Source: A Hyperparameters and Configurations · p. 20
  • The experiments use greedy-style decoding with T=0.0 and top-p=1.0 to match official repository defaults and isolate the effect of the layer used for next-token prediction.The passage states that Table 7 supports this design choice.Source: A Hyperparameters and Configurations · p. 20
  • Benchmark prompts and decoding settings follow the corresponding official repositories, with the model-serving backend switched to a vLLM-hosted OpenAI-compatible endpoint.Table 6 summarizes benchmark-specific decoding and evaluation settings.Source: A Hyperparameters and Configurations · p. 20, Table 6 · p. 22
  • WritingBench uses its leaderboard settings of T=0.7, top-p=0.8, and top-k=20, and scores responses by averaging judge ratings across criteria and queries.The two-stage evaluation uses a judge to score each response on a 1–10 scale against five instance-specific criteria.Source: A Hyperparameters and Configurations · pp. 21–22
  • On GPQA-Diamond with Qwen3.5-35B-A3B, accuracy fell from 82.8% to 80.8% as temperature rose from 0.0 to 1.0, and from 82.8% to 76.3% as p fell from 1.0 to 0.0.The p=1.0, T=0.0 setting improved over final-layer greedy decoding (p=0.0) by +6.5 absolute points.Source: Table 7 · p. 22, A Hyperparameters and Configurations · p. 22

B Three-Phase Structure on More Models

  • Qwen3.5-9B shows a three-phase structure qualitatively identical to Qwen3.5-35B-A3B, including Phase III heterogeneity in the per-token analysis.Qwen3.5-9B has 32 layers (24 DeltaNet and 8 full-attention) and no sparse expert routing; on GSM8K, its norm ratio resurges at layer 32 while IO-CosSim drops there.Source: B Three-Phase Structure on More Models · p. 22, Figure 5 · p. 23

B.1 Layer-wise Dynamics · B.2 Per-Token Entropy Partitioning

Qwen3.5-9B retains the three-phase layer-wise pattern, while its final layer perturbs a larger share of tokens than in 35B-A3B. The appendix examines this pattern through layer-dynamics measurements and per-token entropy partitioning.

B.1 Layer-wise Dynamics

  • Figure 5 reports Qwen3.5-9B-Base layer-wise metrics on GSM8K and preserves the three-phase progression seen in the 35B-A3B analysis.Source: §B.1 · p. 23
  • At layer 1, Norm Ratio is ≈8.5 and IO-CosSim is ≈0.14, indicating that the first layer effectively overwrites the token embedding.Source: §B.1 · p. 23
  • Across layers 5–31, Norm Ratio stabilizes at 0.26–0.58 while IO-CosSim remains 0.88–0.97, consistent with incremental, directionally faithful refinement.Source: §B.1 · p. 23
  • At layer 32, the Norm Ratio rises to ≈0.84 and IO-CosSim drops to ≈0.80, the largest directional deflection in the late regime.This matches the late-layer pattern observed in 35B-A3B.Source: §B.1 · p. 23

B.2 Per-Token Entropy Partitioning

  • Figure 6 applies the per-token entropy partitioning protocol to Qwen3.5-9B on GPQA Diamond, using 50 prompts and 4,096 generated tokens per prompt.Tokens are classified as perturbed when entropy rises at the final layer, or unperturbed when it does not.Source: §B.2 · p. 23
  • On the 9B backbone, 47.4% of tokens are perturbed (mean ΔH=+0.34 nats), compared with only 16.2% on 35B-A3B; the qualitative three-phase pattern is preserved across model scales.Source: §B.2 · p. 23
  • The higher perturbation rate is consistent with a compressed refinement corridor (L=32 vs. L=40): fewer layers complete Phase II refinement, making final-layer overshoot more likely.Despite the rate difference, the qualitative three-phase pattern is preserved across model scales.Source: §B.2 · p. 23

C Token-Level Substitution Analysis

This section examines token-level substitutions made when the entropy-trough layer disagrees with the final layer.

C Token-Level Substitution Analysis

  • Table 8 lists the 20 most frequent substitutions for Qwen3.5-35B-A3B Base and Instruct on GPQA-Diamond, plus categories for all substitution events.The analysis focuses on tokens changed by Confident Decoding relative to the final layer.Source: C Token-Level Substitution Analysis · p. 23

D More Details of the Baselines

Reimplemented DoLa and SLED improve over last-layer decoding on Qwen3.5-35B-A3B but fall short of Confident Decoding; the section attributes their limitations to assumptions about layer-to-layer representational consistency that hybrid and MoE architectures violate.

D More Details of the Baselines

  • For Qwen3.5-9B on GPQA Diamond, 47.4% of tokens show final-layer entropy rise (mean ΔH=+0.34 nats), versus 16.2% on 35B-A3B; unperturbed tokens select l=32.The other 52.6% continue Phase II refinement through layer 32 without disruption.Source: D More Details of the Baselines · p. 24, Figure 6 · p. 24
  • On Qwen3.5-35B-A3B, reimplemented DoLa and SLED improve over last-layer decoding, but their gains remain substantially below Confident Decoding.The re-implementations were validated on LLaMA-family models; Table 9 reports the Qwen3.5-35B-A3B comparison.Source: D More Details of the Baselines · pp. 24–25, Table 9 · p. 25
  • DoLa assumes early and late layers occupy comparable subspaces, while SLED assumes locally smooth, informative logit trajectories; hybrid layer alternation and expert switching undermine these assumptions.DoLa’s contrastive signal is diluted by architectural noise, while SLED’s logit-evolution signal is corrupted by non-monotonic entropy profiles.Source: D More Details of the Baselines · p. 25
  • Both baselines assume representational homogeneity across layers, but hybrid and MoE architectures violate this condition through discontinuous shifts in representation geometry.The original implementations rely on dense-architecture layer-indexing and residual-stream access patterns, so they cannot directly support hybrid or MoE backbones.Source: D More Details of the Baselines · pp. 23, 25
  • Confident Decoding independently evaluates each candidate layer’s predictive entropy and selects the minimum, avoiding cross-layer subtraction that can amplify structured noise.It still uses the shared unembedding projection WU and is therefore not immune to probe mismatch.Source: D More Details of the Baselines · p. 25

E Degradation Analysis

Confident Decoding’s effects vary across backbones and tasks; the analysis links vulnerability to hybrid layer heterogeneity and limited depth, while treating MoE robustness as a plausible but confounded explanation. Its applicability depends on having a sufficiently wide late-refinement corridor for the true entropy gradient to dominate probe noise.

Factor I: Hybrid architecture compresses the refinement corridor

  • Qwen3.5-9B’s hybrid stack has 24 DeltaNet and 8 full-attention layers, compressing the refinement corridor so a one-layer rollback can cross from global-context consolidation to a pre-final state.The backward scan’s shared final unembedding probe may be mismatched across alternating layer types, producing structured, layer-dependent projection noise that can shift the observed entropy valley away from the semantic optimum.Source: E Degradation Analysis · pp. 25–26
  • Probe mismatch is task-dependent: Qwen3.5-9B gains +6.6 on LCB-v6, while degradation concentrates on GPQA-D and Omni-MATH, where shallow entropy gradients are more easily masked by probe noise.The structured noise is insufficient to derail the scan when the entropy-valley signal is strong and concentrated in the final few layers, as in code generation on LCB-v6.Source: E Degradation Analysis · p. 26

Factor II: Model depth widens the refinement corridor

  • Deeper stacks widen the homogeneous late-refinement corridor: Qwen3.5-27B has no GPQA-D degradation (80.0 →80.0) and gains +10.1 on LCB-v6, though depth is confounded with model size and expert routing.The wider corridor provides more consecutive layers for the true entropy gradient to dominate probe noise and for the backward scan to locate a reliable valley.Source: E Degradation Analysis · p. 26

Why MoE architectures appear robust

  • MoE routing may support robustness by concentrating task-relevant updates in specialized experts and smoothing adjacent-layer representations, but its contribution cannot be separated from depth in this model suite.The authors leave isolating depth and MoE routing to future work requiring models that differ on only one of those axes.Source: E Degradation Analysis · p. 26

Scope delimitation

  • Even broadly beneficial Qwen3.5-27B drops −3.0 on Air-Bench (67.0 →64.0); the stated precondition is a sufficiently wide corridor for the entropy gradient to dominate probe noise (∆η(l) > 2ϵmax).The Qwen3.5-9B degradation does not invalidate the entropy-valley hypothesis; deterministic valley selection can fail when a compact hybrid backbone’s corridor is too narrow for universal rollback.Source: E Degradation Analysis · p. 26

Degradation patterns

  • Qwen3.5-9B regresses on GPQA-D (64.6 →62.1) and Omni-MATH (49.1 →47.1), but improves on LCB-v6 (41.1 →47.7), Air-Bench (53.0 →56.0), and HLE (5.2 →6.1).The results show that Confident Decoding does not uniformly benefit all backbones.Source: E Degradation Analysis · p. 25, Table 10 · p. 27

Related papers

Loading 2606.21906v1…