Rewiring Experts on the Fly:Continuous Rerouting for Better Online Adaptation in Mixture-of-Expert models

1 INTRODUCTION

• MoE routing can bottleneck performance because expert selection depends on earlier routing decisions and lacks an inference-time mechanism to reinforce useful routes.Standard approaches such as demonstrations and parallel candidate generation influence routing only implicitly.Source: §1 · pp. 1–2
• Existing test-time training approaches may use retrieved reference data, while the proposed framework treats each prompt as a self-supervised learning opportunity.The related approach using successful neighbors incurs retrieval overhead and can fail with short prompts; it is also static as generation proceeds.Source: §1 · p. 2
• The framework alternates context-based routing optimization with steered generation, using only the current context and lightweight additive vectors on selected router logits.It uses backpropagation without external data or retrieval, and limits updates to selected high-confidence layers to reduce overhead and prevent over-adaptation.Source: §1 · p. 2
• The authors report improved performance on complex reasoning tasks, robustness to context shifts in multi-turn conversations, and compatibility with existing test-time techniques.They describe routing changes as lightweight and quick to compute and apply, including in multi-user settings.Source: §1 · p. 3

Mixture-Of-Experts (MoE)

• MoE language models use sparse activation and routing to scale model capacity while limiting computation and memory costs.Top-k routing activates only a subset of experts for each input token.Source: §2 · p. 3

Test-Time Scaling

• Test-time scaling allocates additional inference computation through parallel generation or sequential extensions such as chain-of-thought reasoning.Examples include self-consistency, best-of-N sampling, Monte Carlo Tree Search, and extending outputs.Source: §2 · p. 3

Test-Time Training (TTT)

• Test-time training and reinforcement learning approaches adapt language models using retrieved neighbors, optimized data selection, or majority-vote rewards.For MoE routing, Li et al. (2025a) use successful neighbors from reference sets, assuming deployment data access and adding retrieval overhead.Source: §2 · p. 3

3.1 PRELIMINARIES ON MIXTURE-OF-EXPERTS

• The method uses a router and multiple experts in each MoE layer to assign input tokens to a subset of experts.The router computes expert-selection logits and probabilities, then a routing strategy such as top-k selects active experts.Source: §3.1 · pp. 3–4

3.2 ROUTER LOGITS MODIFICATION

• Layer-specific additive parameters modify router logits, changing expert-selection probabilities according to prompt characteristics.The modified logits are the original logits plus an adjustment vector for the selected layer.Source: §3.2 · p. 4

3.3 DYNAMIC LAYER SELECTION

• The framework updates only selected MoE layers to reduce computation and prevent over-adaptation, prioritizing layers with higher routing confidence.Confidence aggregates token-level top-expert probabilities; higher values indicate more decisive routing that may be more task-relevant.Source: §3.3 · p. 4

3.4 OPTIMIZATION PROCEDURE

• The optimization procedure selects layers from the current context and performs cross-entropy updates to their routing parameters.An optimizer such as SGD or Adam can update the parameters.Source: §3.4 · pp. 4–5
• After optimization, the model generates a batch of tokens with the updated routing parameters, then repeats optimization using the extended context.The context includes the original prompt and all generated tokens so far.Source: §3.4 · p. 5

• Evaluation covers MMLU-redux, HumanEval, MBPP-sanitized, GSM8K, and MATH500 across general knowledge, code generation, and mathematical reasoning.MMLU-redux uses generation rather than multiple-choice format.Source: §4 · p. 5
• The baselines are ICL with 3 and 5 sample pairs, respectively, and C3PO with 100 reference samples per dataset.Parallel-generation methods are treated as an orthogonal approach.Source: §4 · p. 5
• Experiments use OLMoE, Qwen1.5-MoE, and DeepSeek-V2-Lite, with different layer and expert configurations.The models have 16, 60 routing, and 64 routed experts per layer, respectively; DeepSeek-V2-Lite also has 28 layers.Source: §4 · p. 5
• Adaptation uses Adam for 5 iterations with zero-initialized parameters, and re-optimizes every 128 generated tokens.The learning rate is 0.05, with soft-weighted layer selection.Source: §4 · p. 5

5.1 MAIN RESULTS

• Across five benchmarks, rewiring consistently beats all baselines; HumanEval gains are 3.6% on DeepSeek-V2-Lite, 5.5% on OLMoE, and 6.7% on Qwen1.5-MoE.The method also outperforms few-shot learning and C3PO without using example demonstrations or reference data.Source: §5.1 · p. 5, Table 1 · p. 6

5.2 ABLATIONS OF KEY COMPONENTS

• Ablations across five tasks examine layer-selection strategies and continuous versus non-continuous refinement.Source: §5.2 · p. 6, Table 2 · p. 6
• Confidence-based layer selection averages 53.59%, above random selection (49.77%), last-five-layer selection (50.72%), and reverse selection (49.35%).The authors interpret router confidence as identifying layers with strong expert-selection knowledge.Source: §5.2 · p. 6, Table 2 · p. 6
• Updating all layers averages 50.68%, below selective updates at 53.59%; the authors say spreading limited optimization steps risks diluting refinement and destabilizing pretrained routing.Source: §5.2 · p. 6, Table 2 · p. 6
• Continuous refinement averages 53.59%, compared with 52.55% for non-continuous refinement, a gap of over 1%.The authors attribute the improvement to gradual refinement of expert selection during test-time optimization.Source: Table 2 · p. 6, §5.2 · p. 7

6 ANALYSIS AND DISCUSSIONS

• Pathway edits peak in deeper layers (HumanEval 18–22, >0.35; MATH500 20–22, up to 0.16), while activation shifts target subsets and differ by task.The activation shifts are stronger in later layers L21–L25.Source: Figure 2 · p. 7, §6 · p. 7
• Across 18 generation steps, rerouted router entropy gradually falls while baseline entropy remains higher and fluctuates, indicating increasingly focused expert choices.Entropy is measured across layers and averaged over 16-token blocks.Source: Figure 3 · p. 8, §6 · p. 8

6.3 EFFICIENCY ANALYSIS

• 1.3× fewer FLOPs than C3PO and 1.6× fewer than ICL (5-shot), with 20.12s inference time per sample on HumanEval using DeepSeek.The method requires more computation than the vanilla baseline but is more efficient than most compared test-time approaches.Source: Table 3 · p. 8, §6.3 · pp. 8–9

6.2 COMBINING REWIRING WITH OTHER TEST-TIME STRATEGIES

• Combining rerouting with 3-shot ICL improves multiple tasks, even over rerouting alone.The comparison is reported across multiple tasks in Table 4.Source: §6.2 · p. 8, Table 4 · p. 9

6.4 EXTENSION TO LONG-REASONING TASKS

• 75.65% vs. 74.29% on AIME25 and 73.75% vs. 69.58% on AIME24 in average correctness; improvements occur chiefly in Maj@k rather than Pass@k.The comparison uses GPT-OSS-20B.Source: Table 5 · p. 9, §6.4 · p. 9

6.5 ROBUSTNESS TO CONTEXT SHIFTS IN MULTI-TURN SCENARIOS.

• Across HumanEval and Math500, rerouting outperforms baselines under both aligned-task and shifted-task few-shot contexts.Shifted-task examples come from unrelated domains.Source: Figure 4 · p. 10, §6.5 · p. 10

7 CONCLUSION

• Up to 6.7% gains on code generation and 1.6× fewer FLOPs than few-shot methods accompany consistent benchmark gains and context-shift robustness.The conclusion also reports that rerouting combines with complementary techniques such as self-consistency.Source: §7 · p. 10
• The method alternates routing optimization and steered generation, using lightweight additive vectors to update selected router logits and reduce computational overhead.The conclusion describes this cycle as a feedback loop for progressively improving expert selection.Source: §7 · p. 10
• As a plug-and-play strategy, rerouting can combine with complementary techniques and points toward deployment-time adaptation and continual self-regulation in MoE models.Source: §7 · p. 10

A.1 BENCHMARKS

• The benchmark suite includes MMLU-Redux, HumanEval, sanitized MBPP, GSM8K, and MATH-500.MMLU-Redux is manually re-annotated, and this paper uses the sanitized MBPP split.Source: §A.1 · pp. 13–14

A.2 BASELINES

• C3PO is the primary pathway-optimization baseline, estimating test pathways by kernel regression over correct-prediction reference examples.The reference sets use 100 randomly sampled instances on which the model predicts correctly.Source: §A.2 · p. 14
• ICL is also evaluated using 3–5 randomly sampled, task-aligned demonstrations prepended to each query.Demonstration sources vary by benchmark, including MBPP prompts, GSM8K training examples, and MMLU validation examples.Source: §A.2 · pp. 14–15

A.3 MODEL SELECTION

• The evaluation uses OLMoE, DeepSeek-V2-Lite, Qwen1.5-MoE, and GPTOSS-20B.The models span differing expert configurations and active parameter counts.Source: §A.3 · p. 15

B.1 TEST-TIME MOE REROUTING

• Algorithm 1 lays out the test-time procedure for adaptive expert selection and pathway calibration.It is presented as a step-by-step rerouting pipeline.Source: §B.1 · p. 15

B.2 PATHWAY DIFFERENCES

• An input’s pathway is represented as the ordered sequence of experts selected across its MoE layers.The sequence is formed by joining the selected expert indices across layers.Source: §B.2 · p. 16
• Pathway differences are measured with Levenshtein edit distance, capturing mismatches in expert selection.Expert indices are formatted as comma-separated strings per layer and joined across layers with hyphens.Source: §B.2 · p. 16

C.1 LAYER-WISE CONFIDENCE DISTRIBUTIONS ACROSS DIFFERENT TASKS

• Math500 confidence peaks in middle layers 7–14, whereas HumanEval confidence is more distributed, with peaks in layers 15–18.Both tasks show higher confidence concentrated in middle-to-later layers than in early layers.Source: §C.1 · p. 16, Figure 5 · p. 16

C.2 SENSITIVE ANALYSES OF HYPER-PARAMETERS

• Optimization intervals of 128–160 tokens achieve optimal performance across five benchmarks, while 96-token updates and no updates degrade performance.The passage attributes the degradation to over-adaptation with frequent updates and loss of dynamic adjustment benefits without updates.Source: §C.2 · p. 16
• Mathematical reasoning tasks GSM8K and MATH500 benefit most from tuning the optimization interval.Figure 6 compares interval settings across five benchmarks against the no-rerouting baseline.Source: §C.2 · p. 16

C.3 IMPACT ON SAMPLE DIVERSITY IN PARALLEL GENERATION.

• Cosine diversity scores are nearly identical across conditions (0.380±0.150 vs. 0.390±0.157), as are semantic diversity scores (0.012±0.008 vs. 0.012 ± 0.007).The authors report that online adaptation maintains comparable diversity to the baseline.Source: Table 6 · p. 16, §C.3 · p. 17
• The evaluation uses DeepSeek MoE with default settings to generate 10 HumanEval samples.The authors compare parallel-generation sample diversity under the method and baseline.Source: §C.3 · p. 17
• Diversity is assessed with CodeBERT embedding-based semantic similarity and TF-IDF cosine diversity.The measures capture functional similarity between code snippets and lexical variation, respectively.Source: §C.3 · p. 17