Source-linked AI summary
Router Prior Bias: Preserving Base Routing Structure in MoE Post-Training
Jaedeok Lee, Keonwoo Kim, Dongyoon Han, Sangdoo Yun, Yera Choi, Haanju Yoo
TL;DR
MoE post-training must preserve the base router’s learned non-uniform structure rather than re-impose uniform utilization, but prior defaults confound what to preserve with how strictly to enforce it. The paper introduces soft router anchoring, including RPB, and finds that soft enforcement outperforms re-applied LBL across model families and corpora, while hard enforcement preserves routing structure without the same performance benefit.
Problem
MoE pretraining induces non-uniform expert specialization and co-activation patterns, yet post-training often re-applies uniformity objectives or leaves routing unconstrained, leaving the appropriate routing constraint unresolved.
Method
The paper evaluates soft router anchoring, including Router Prior Bias (RPB), and measures retention of expert utilization, token-level top-k selection, and co-activation community structure.
Results
Re-applied LBL degrades downstream performance relative to soft anchoring across model families and corpora, while hard constraints preserve community structure but substantially reduce performance.
Takeaways & Limitations
Post-training should hold inherited routing softly rather than flattening it toward uniformity or enforcing it absolutely; community structure is a footprint of soft enforcement, not its source.
Takeaways & Limitations
The advantage over a no-load-balancing baseline is model-dependent, and community-level interpretations apply only when the base router carries community structure.
Abstract
from arXiv · showhide
Mixture-of-Experts (MoE) pretraining relies on an auxiliary load-balancing loss (LBL) to drive per-expert utilization toward uniformity. Post-training inherits a different situation: the base router already encodes non-uniform expert co-activation structure, which a re-imposed uniformity objective flattens away. We show that downstream performance depends instead on holding this inherited routing softly, a principle we term soft router anchoring, and instantiate it as Router Prior Bias (RPB), a training-time bias that pulls the router logits toward a prior read off the frozen base router while leaving the router itself trainable. On math post-training of Moonlight-16B-A3B, RPB attains 45.77 in-domain accuracy against 31.91 under re-applied LBL and 29.44 under unanchored fine-tuning, and retains more out-of-domain capability than either. The ordering against LBL reproduces on a second model family (Qwen3-30B-A3B-Base), and the advantage over LBL is resolvable on an independently sourced corpus. Anchors defined on the router weights, on its logits, or on its output distribution perform comparably with no consistent ordering, which places the effect in the softness of the constraint rather than in the particular prior RPB supplies. Retained community structure in the expert co-activation graph tracks these gains wherever the base router is non-uniform enough for communities to form, yet enforcing the same prior as a hard assignment preserves that structure while performance falls sharply. Community structure is therefore a footprint of soft anchoring rather than its source, and the practical lesson is that inherited routing should be held softly during post-training, since both flattening it toward uniformity and enforcing it absolutely carry a downstream cost. Our code will be released at https://github.com/naver-ai/rpb.
1 Introduction
MoE pretraining promotes uniform expert utilization, but post-training begins with specialized, non-uniform routing that re-applied load balancing can disrupt. The paper introduces soft router anchoring, including RPB, to preserve inherited routing while keeping the router trainable.
- Motivation: Pretraining induces non-uniform expert specialization and recurring co-activation patterns, making uniformity-based post-training potentially misaligned.LBL is designed to penalize deviations from uniform per-expert utilization.
- Soft router anchoring: RPB anchors routing to a frozen-base prior through a trainable, continuous-strength logit bias.This separates the routing structure being preserved from how strictly it is enforced.
- Retention: Soft anchoring retains base top-k selection and community structure while re-applied LBL primarily preserves per-expert utilization.Direct router-parameter objectives behave similarly, indicating anchoring—not RPB’s particular prior—is the shared factor.
- Causal controls: Hard enforcement preserves community structure yet substantially reduces performance, separating soft enforcement as the source of gains from community preservation as its footprint.Shuffling prior content at fixed enforcement supports the same distinction.
- Scope: The advantage over unanchored fine-tuning is model-dependent, while the ordering against re-applied LBL persists across model family and corpus.The paper reports both directions for the comparison with the no-load-balancing baseline.
- Diagnostic: Community NMI separates soft anchoring from LBL only when the base router is sufficiently non-uniform to form communities.Moonlight satisfies this condition, whereas DeepSeek-V2-Lite does not.
2 Preliminaries
The paper evaluates routing retention at marginal utilization, token-level expert selection, and expert co-activation community levels. These measurements are paired with downstream accuracy and computed without the training-time RPB bias.
- Retention levels: Routing retention is measured through per-expert utilization, token-level top-k selection, and expert co-activation community structure.All three are reported alongside in-domain and out-of-domain performance.
- Evaluation protocol: Evaluation removes the training-time RPB prior, so measured routing reflects the trained router itself.This bias-free protocol applies to top-k overlap and community structure.
- Per-expert utilization: Per-expert utilization compares base and post-training expert distributions using TVD and JSD.This is the level targeted by classical LBL and serves as a sanity condition rather than the main result.
- Token-level selection: Top-k overlap measures the normalized intersection of base-router and post-training expert selections across tokens and layers.It captures token-level selection identity rather than only marginal utilization.
- Community structure: The co-activation graph treats experts as nodes and joint top-k selection frequency as weighted edges.Graphs are constructed layer-wise from experts appearing together in token routes.
- Community comparison: Louvain partitions each graph into communities, and community NMI compares post-training partitions with the base partition.Edges are thresholded by retained-edge fraction ρ before community detection.
3 Method
RPB forms frozen-base routing priors over prompt and answer spans, converts them into centered logit biases, and applies them during training before top-k selection. The method is compared with hard enforcement, router-weight anchoring, and output-space anchors.
- Prior construction: RPB precomputes a frozen-base routing prior separately for each sample, Q/A span, layer, and expert.Q and A are separated because they induce different base routing distributions.
- RPB: The prior becomes a zero-mean log-prior bias added to current router logits during training before top-k selection.The bias is broadcast across the relevant span and controlled by β.
- Training and evaluation: RPB removes the bias at evaluation, leaving the trained router to determine measured routing.Only the current model parameters train; the prior tensor remains frozen.
- Hard control: RPB-hard uses the same prior as the sole top-k score, forcing span-level expert selection and removing token-conditioned routing.It provides the hard-enforcement control for the intervention-strength axis.
- Router-L2: Router-L2 anchors router weight matrices to their frozen-base values while leaving experts, attention, and MLP blocks trainable.Unlike RPB, it uses no sample-conditional prior or span labels.
- Output-space anchoring: Output-space anchors constrain current router probabilities or centered logits toward the frozen-base prior through training losses.Unlike RPB, these anchors do not enter the top-k selection step.
4 Experimental Setup
Experiments compare routing interventions across three MoE checkpoints, two post-training domains, and a broad benchmark suite. The main evaluation contrasts unanchored fine-tuning, LBL, and several base-router-preserving methods.
- Models: The study examines Moonlight-16B-A3B, DeepSeek-V2-Lite, and Qwen3-30B-A3B-Base.Moonlight and DeepSeek share expert geometry but differ in router scoring, while Qwen supplies a second family for performance comparison.
- Interventions: The main conditions compare SFT/NoAux, marginal load balancing, multiple anchoring mechanisms, and controls that vary enforcement strength or prior content.All conditions share the experimental protocol and differ in their auxiliary objective or router-attached bias.
- Router characteristics: Moonlight has strongly non-uniform, domain-divergent routing, whereas DeepSeek-V2-Lite has diffuse routing and is unsuitable for the same community diagnostic.Qwen is used for performance claims rather than community analysis.
- Datasets: math20k and coding20k provide paired in-domain and out-of-domain post-training settings.Each 20,000-row split comes from GLM-5.1-Reasoning-1M-Cleaned and uses prompt-answer pairs.
- Evaluation: Evaluation covers twelve benchmarks spanning math, coding, general and professional QA, STEM, and general reasoning.Results report AvgID and AvgOOD, with aggregate scores using a nine-benchmark subset.
5 Results
Across model families and corpora, soft anchoring consistently outperforms re-applied LBL, while its advantage over unanchored fine-tuning is model- and corpus-dependent. Routing diagnostics and intervention controls indicate that community retention is a footprint of soft enforcement, not the performance source.
- Main results: 45.77 in-domain accuracy: RPB exceeds LBL at 31.91 and SFT at 29.44 on Moonlight-16B-A3B math20k.RPB also reaches 19.53 out-of-domain accuracy, versus 14.97 for LBL and 15.65 for SFT.
- Anchoring forms: Four anchoring objectives perform comparably across spaces, with no consistent winner across training corpora.The result indicates that holding routing near its base state matters more than the particular anchoring space or objective.
- Generality: Re-applied LBL remains the weakest condition on Qwen3-30B-A3B-Base and OpenR1-Math-220k, but anchoring does not consistently beat no-load-balancing fine-tuning.On Qwen3, RPB leads LBL by 2.64 points on math20k, whereas coding20k differences fall within seed noise; on OpenR1-Math-220k, LBL trails RPB by 10.20 Overall@9 points.
- Routing retention: Per-expert utilization does not distinguish methods, while community-level retention separates soft anchoring from the alternatives.Top-k overlap provides weaker separation, and all methods occupy similar TVD and JSD ranges.
- Soft versus hard enforcement: Within-soft remains near RPB, whereas within-hard sharply reduces performance despite preserving community membership.Global-hard reduces performance further while achieving the highest community NMI, showing that retained community structure alone is insufficient.
- Scope condition: Community NMI is a diagnostic footprint rather than an optimization target, and it is informative only when the base router has sufficiently non-uniform community structure.DeepSeek-V2-Lite has nearly uniform routing, so NMI fails to separate methods even though RPB improves performance and the soft-versus-hard dissociation remains.
6 Discussion
The discussion argues that post-training should preserve inherited expert co-activation structure softly rather than target marginal load or enforce routing absolutely. It cautions that retention metrics are useful within their calibrated range, but optimizing them directly can reduce performance.
- Interventional controls identify enforcement strength, rather than preserved community structure, as the source of the performance gain.
- Hard enforcement can preserve community structure while sharply reducing performance, separating routing retention from the mechanism producing gains.
- A distance-from-base proxy is informative only within its calibration range; directly optimizing it can push the system beyond that range.
- 15.2 points on math20k and 6.9 on coding20k are lost when expert identities are shuffled across communities rather than within them under fixed hard enforcement.
- Soft anchoring should target inherited co-activation structure rather than per-expert load, because flattening marginal utilization removes properties supporting out-of-domain retention.
7 Related Work
Related work frames MoE routing around pretraining load balancing, expert specialization, and post-training choices to drop balancing or freeze routing. This paper instead constrains routing changes while preserving trainability.
- MoE pretraining commonly uses auxiliary load-balancing objectives to stabilize expert utilization, while refinements include global-batch, auxiliary-loss-free, and dynamic-bias approaches.
- Practitioner toolchains often drop load balancing or router training during post-training, whereas this work constrains how routing changes instead of reshaping the expert pool.
- Related expert-specialization work includes shared and fine-grained routed experts, while routing probes report semantic and multilingual signals.
8 Conclusion
The paper finds that soft anchoring preserves inherited routing better than re-applied load balancing, while the anchoring space itself is not decisive. The conclusion is bounded by model dependence against unanchored fine-tuning and by the need for suitable routing structure.
- Re-applied load balancing degrades downstream performance relative to soft anchoring across model family and training corpus changes.
- Four anchoring objectives acting in different spaces perform comparably, making RPB one implementation of soft router anchoring rather than a uniquely necessary method.
- The advantage over a no-load-balancing baseline is model-dependent, so soft anchoring is not uniformly superior to unanchored fine-tuning.
Limitations
The evidence is bounded by model, corpus, objective, and training-regime scope. In particular, community-level interpretations and comparisons with unanchored fine-tuning do not generalize uniformly across settings.
- Model and diagnostic scope: The community diagnostic is scope-limited: it applies clearly to Moonlight-16B-A3B but not to diffuse-routing DeepSeek-V2-Lite or the Qwen3 performance comparison.Broader applicability would require models spanning routing sharpness and additional MoE families.
- Baseline dependence: Soft anchoring versus unanchored fine-tuning is model- and corpus-dependent, with reversal on Qwen3 coding20k.The two methods are indistinguishable on Qwen3 math20k, while unanchored fine-tuning is superior on coding20k.
- Data scope: The experiments use controlled 20k-example splits from one source corpus, so they do not establish effects under larger or more heterogeneous instruction mixtures.The limitation concerns both hyperparameter ranges and effect sizes.
- Objective scope: The anchoring study does not determine which prior target is optimal or predict which anchoring form will lead on a given corpus.Each objective fixes a target while varying strength, and ordering changes across corpora.
- Training-regime scope: The study covers non-reasoning base models and QA splits, not long-chain-of-thought post-training with sequential routing dynamics.Long-form generation may require transition-community analyses rather than static co-activation graphs.
Appendices
Appendix experiments examine anchoring strength, generic router regularization, community validity, and detection robustness. Together they show that soft strength matters, generic regularization is insufficient, and the community diagnostic is methodologically stable but not a complete explanation.
- Anchoring-strength sweeps: Community NMI saturates by β=0.5, while performance peaks near β=0.75 to 1.0, separating retention from downstream optimization.The hard RPB endpoint slightly raises NMI but reduces performance on both corpora.
- Anchoring-strength sweeps: Router-L2 varies performance only slightly across two orders of magnitude, with the best anchoring strength reversing between math20k and coding20k.The three settings span 0.25 points on math20k and 1.0 point on coding20k.
- Regularization baseline: The ST-MoE z-loss falls below RPB in every comparison, showing that generic router-logit regularization does not substitute for anchoring toward inherited routing.On Moonlight it also falls below unanchored fine-tuning on both corpora.
- Community validation: Observed co-activation graphs exceed marginal-preserving null graphs on modularity in 100% of cells, validating the community object without making it a sufficient explanation.The modularity margins are consistently larger than the NMI margins.
- Detection robustness: Changing edge thresholds preserves the method ordering, while Leiden agrees with Louvain within 0.009 on every cell.Detector-seed dispersion averages 0.031 and never exceeds 0.050.
C Base-Router Heterogeneity Probes
The heterogeneity probes compare routing structure across MoE families and test ancillary router regularization. They support cross-family structural differences while preserving the paper’s distinction between routing diagnostics and downstream performance.
- Qwen3 probe design: Qwen3-30B-A3B-Base is probed as a frozen non-reasoning router, with Q/A spans analyzed across math, code, and general clusters.The appendix uses this model to assess whether the Q/A split is specific to Moonlight.
- Strength and hardness: The operating-range sweep finds community NMI saturating before peak performance, while hard enforcement raises NMI slightly but lowers downstream scores.This supports treating community retention as distinct from the performance-optimal softness level.
- Alternative anchor: Router-L2 provides a weight-space anchoring comparison whose best setting depends on the corpus and whose weaker coding20k result is single-seed.The λ=104 rows are three-seed means, whereas weaker settings use one seed.
- Generic regularization: The ST-MoE z-loss is a non-anchoring baseline that penalizes large router logits without referencing the base router.Its results are reported on a twelve-benchmark mean at one training seed.
- Q/A routing structure: Q tokens separate math, code, and general probe clusters more sharply than A tokens in both Moonlight and Qwen3.The split used by RPB therefore appears across two model families.
- Community null test: Marginal-preserving null comparisons show real co-activation graphs outperforming nulls on modularity in every cell, while margins do not order methods like performance.The check validates the community object rather than establishing it as the performance mechanism.
D.1 Cross-Architecture Robustness Check
The cross-architecture check separates performance robustness from community-diagnostic robustness. DeepSeek-V2-Lite retains a soft-versus-hard performance pattern despite diffuse routing, but community NMI cannot distinguish methods there.
- Cross-architecture setup: DeepSeek-V2-Lite has more diffuse routing than Moonlight, enabling separate tests of performance and community-level evidence.Its performance claim and community diagnostic are not assumed to rise or fall together.
- Community diagnostic: On DeepSeek-V2-Lite, all four fine-tuning conditions cluster near community NMI ≈0.49, so NMI does not separate methods.The near-uniform graph structure limits the diagnostic’s discriminative value.
- Soft versus hard control: Soft token-conditioned routing remains important without community separation: hard span-level control drops overall math20k performance from 16.12 to 9.44 and coding20k from 22.58 to 16.92.In-domain math20k accuracy also falls from 22.88 to 9.49.
- Interpretive boundary: Across models, the soft-versus-hard pattern is consistent, whereas the community-mediated interpretation is model-dependent and not general beyond the tested models.Qwen3 preserves the LBL ordering without community-level evidence serving as the operative explanation.
- Routing heterogeneity: DeepSeek-V2-Lite’s base routing is closer to uniform and less cross-domain divergent than Moonlight’s, with L1 sharpness 0.1399 versus 0.5445 and cross-domain divergence 0.0724 versus 0.3735.These differences are presented as candidate explanations, not experimentally isolated causes.
- Control design: The intervention controls distinguish within-community from global shuffles and soft from hard enforcement, with prior-side measurements taken before training.Top-k overlap, effective bias change, and clamp activation quantify intervention strength rather than trained-model routing outcomes.
- Evaluation protocol: The cross-architecture table is based on a single training seed and aggregates twelve benchmarks, limiting direct recomputation against nine-benchmark main-table results.The protocol difference should be considered when comparing aggregate scores.
- Retention measures: The retention hierarchy finds community NMI separating soft anchors from SFT and LBL, while TVD and JSD remain in the same range across methods.Top-k overlap places SFT between the two anchors on coding20k.
G.3 Per-Benchmark Shuffled-Prior Controls
The shuffled-prior controls compare soft and hard anchoring at aggregate and per-benchmark levels across Moonlight-16B-A3B and DeepSeek-V2-Lite. Moonlight shows a consistent advantage for soft controls over hard controls, while the DeepSeek comparison reproduces the soft-versus-hard gap.
- Aggregate and per-benchmark evaluation: The Moonlight comparison is reported both as aggregate Overall@9 performance and community NMI for each condition.Per-benchmark results are provided alongside the matched unshuffled RPB row for column-by-column checking.
- Moonlight-16B-A3B: RPB ≈within-soft ≫within-hard > global-hard on Moonlight-16B-A3B across both fine-tuning splits.The ordering holds over the three training seeds used elsewhere in the paper.
- DeepSeek-V2-Lite: On DeepSeek-V2-Lite, the soft-versus-hard performance gap reproduces, while global-hard is slightly higher than the other hard-control ordering described.The comparison is presented through the four main interventions and shuffled-prior controls.
- Evaluation coverage: The per-benchmark tables cover math20k and coding20k, with results grouped by fine-tuning dataset and benchmark family.Moonlight results use three training seeds and avg@5 evaluation repeats, while DeepSeek-V2-Lite uses five evaluation repeats at a single training seed.