PaperlayerView exact revision on arXiv

Source-linked AI summary

DuoRate: Two-Rate Per-Expert Precision Allocation for Mixture-of-Experts Serving

Mengqi Guo, Zhe Wei, Yuan Yuan, Jiunn Bin Lim, Boyi Pan, Michael Bi Mi

arXiv:2610.13328v1cs.LG

TL;DR

MoE precision sets both resident storage and batch-one decode traffic, yet published per-expert allocators constrain only storage. DuoRate stores one prefix-readable tier and allocates workload-specific decode reads; on Qwen3-30B-A3B, it outperforms 2-bit-class methods at 2.3 bpw and matches 4-bit formats at 2.9 bpw. Its evaluation covers one MoE model, and read-side savings still need end-to-end measurement with a fused prefix-read kernel.

  • Problem

    MoE allocators constrain resident precision but not routing-weighted decode traffic, even though the same expert weights incur separate storage and batch-one read costs.

    Source: §1 · p. 1

  • Method

    DuoRate stores one tier of a prefix-readable stream and uses label-free workload prefill statistics to allocate per-expert decode prefixes under a rate budget.

    Source: §1 · p. 3

  • Results

    At 2.3 bpw, DuoRate outperforms 2-bit-class methods; at 2.9 bpw, it matches 4-bit formats, and on long decoding it matches or exceeds them while staying within a quarter point of BF16.

    Source: §1 · p. 2, §6 · p. 10

  • Takeaways & Limitations

    MoE efficiency claims should report both stored and read rates, because a checkpoint that is cheap to store need not be cheap to decode.

    Source: §6 · p. 10

    The evaluation covers one MoE model, and end-to-end read-side savings with a fused prefix-read kernel remain to be measured.

    Source: §5 · p. 9

Abstract

from arXiv · show

Serving a Mixture-of-Experts (MoE) model pays for expert precision twice: the resident footprint pays for every expert the router can reach, while each batch-one decode step pays only for the experts it selects. These are two different rate functionals over the same per-expert widths, yet published per-expert MoE allocators constrain only the first. We call the difference the store-read gap; in the released BitsMoE checkpoint, 2.54 bits per weight (bpw) of storage become 7.50 bpw of decode traffic. We propose DuoRate, which stores once and reads per workload. Offline, it stores one uniform tier of a prefix-readable weight stream and bakes a distortion tensor. Online, it profiles a workload's own prefill without labels and solves a Lagrangian rate-distortion allocation under the decode functional; routing frequency provably cancels from every per-expert decision, and the solve costs one dot product per expert and a bisection. A deployment therefore keeps a single artifact that prefill reads at the stored rate and decode reads at the workload's own rate. On Qwen3-30B-A3B, DuoRate reading 2.3 bpw exceeds every 2-bit-class baseline by 6.3 to 47.0 points of average accuracy, and at 2.9 bpw it matches 4-bit formats that read 1.4-1.6 bpw more. The advantage is largest under long decoding: on GSM8K, DuoRate at 2.9 bpw matches or exceeds every 4-bit format and trails BF16 by only 0.25 points, and at 2.6 bpw it still matches or exceeds the 4.5-bit NF4 and FP4. Against controls in its own format, the allocation alone is worth 0.07-0.11 bpw of read rate. Code will be released upon publication.

1 INTRODUCTION

MoE precision has separate resident-storage and decode-traffic costs, but existing per-expert allocators constrain only storage. DuoRate allocates decode reads per workload from one stored artifact, and reports stronger accuracy than 2-bit-class baselines while matching 4-bit formats at lower read rates.

1 INTRODUCTION

  • Existing per-expert allocators constrain resident precision, not routing-weighted decode traffic; BitsMoE stores 2.5425 bpw but reads 7.5008 bpw at batch-one decode.The FP16 basis is amortized across 128 experts in storage but only the 8 selected experts at decode, making read rate 2.95× stored rate.Source: §1 · p. 1
  • Routing frequency cancels from each within-domain decode allocation, so it cannot reorder experts at a given budget.DuoRate’s same-format controls attribute 0.07–0.11 bpw of read-rate savings to allocation itself, with every rate-matched comparison excluding zero.Source: §1 · p. 2
  • DuoRate stores one uniform tier and distortion tensor offline, then uses label-free workload prefill statistics and a Lagrangian allocation to select per-expert read prefixes under a budget.A single artifact serves both phases: prefill reads the stored depth, while decode reads the selected prefixes; a new workload needs a read table, not a new checkpoint.Source: §1 · p. 2, Figure 1 · p. 2
  • At 2.3 bpw, DuoRate exceeds every 2-bit-class baseline by 6.3 to 47.0 points of average accuracy; at 2.9 bpw, it matches 4-bit formats reading 1.4–1.6 bpw more.On long-decoding GSM8K, it matches or exceeds every 4-bit format at 2.9 bpw and stays within 0.25 points of BF16.Source: §1 · p. 2

2 RELATED WORK

Prior work studies expert quantization, phase-dependent serving, and rate–distortion allocation, but does not jointly constrain resident storage and routing-weighted decode rate. DuoRate combines a prefix-readable format with a decode-side rate–distortion objective.

Quantizing MoE experts

  • MoE-specific mixed-precision allocators use distortion surrogates and bit budgets, but their constraints are unweighted rates and they report one bits-per-weight figure.The cited methods include block-, layer-, and model-wide allocations, as well as shared-basis and modality-aware approaches.Source: §2 · p. 3

Phase asymmetry in MoE serving

  • Serving research recognizes phase asymmetry and proposes phase-dependent precision, but existing systems use heuristics or cache policies rather than a rate constraint.Prefill loads experts once and amortizes them over routed tokens, while decode touches only a few experts per step; batching and chunked prefill alter these patterns.Source: §2 · p. 3

Rate–distortion allocation and multi-precision formats

  • Rate–distortion bisection and prefix-readable multirate codes are established separately, but prior MoE serving integrations of such codes lack a rate–distortion objective.PrismQuant’s mixture-component weight cancels from rate and distortion at stationarity, a result DuoRate builds on for routing frequency.Source: §2 · p. 3

3 METHOD

DuoRate separates resident storage from batch-one decode reads, then allocates each workload’s read tiers from a single prefix-readable artifact. Its per-workload allocation cancels routing frequency from tier choices, while pooled storage allocation can retain frequency through two identified channels.

3 METHOD

  • StagQ provides tiers p2 through p6 as byte prefixes, with each tier’s rate known and tier changes requiring no re-encoding.Each weight uses a two-bit base code and up to four refinement planes.Source: §3 · pp. 3–4

3.1 TWO RATE FUNCTIONALS, AND WHERE ROUTING FREQUENCY SURVIVES

  • Resident rate counts the experts reached during prefill, whereas batch-one decode rate weights expert reads by routing frequency; as batch size grows, the read functional returns toward an unweighted form.With uniform precision and no shared components, the two rates coincide; shared layer bases can make read rate exceed stored rate, while prefix-readable streams allow it to be lower.Source: §3.1 · p. 4
  • For a single workload, routing frequency cancels from each expert’s tier minimizer and cannot reorder experts; it only selects which allocation meets the budget.Experts’ distortion curves determine promotion thresholds, while the budget determines where the multiplier sweep stops.Source: §3.1 · pp. 4–5
  • Pooled calibration retains routing frequency through the stored-rate bit-width cost and the per-expert mixture of domains; those are its only two assignment channels.The total frequency cancels, but its distribution over domains remains because it weights the curves being averaged.Source: §3.1 · p. 5

3.2 DUORATE: STORE ONCE, READ PER WORKLOAD

  • DuoRate stores a uniform p4 tier at 4.2704 bpw and bakes a workload-independent distortion tensor, leaving every read tier as a prefix of one resident artifact.Its distortion weights squared prefix error by activation energy and output-channel decision sensitivity; the authors leave pooled stored-table allocation to future work.Source: §3.2 · p. 5
  • DuoRate profiles a workload’s prefill without labels, scores each expert and tier with one dot product, then uses a shared-multiplier bisection to meet the read budget.The selected tier is capped by the stored tier; where tier curves are convex, the Lagrangian sweep is globally optimal at every rate it reaches.Source: §3.2 · p. 6
  • At serving time, prefill reads the artifact’s full stored depth, while each decode step reads only selected experts’ assigned prefixes; changing workloads replaces only the small read table.The read table can be recalibrated online from the live stream without labels or a reference model.Source: §3.2 · p. 6

4 EXPERIMENTS

The experiments compare DuoRate with quantization baselines and same-format allocation controls, finding strong accuracy at lower decode-read rates and a measurable gain from allocation. They also show how the preferred allocation and value of reading below the stored rate depend on the deployment setting.

4 EXPERIMENTS

  • The evaluation quantizes 6,144 expert matrices in Qwen3-30B-A3B and compares five 4-bit formats, four 2-bit methods, BitsMoE, and same-format allocation controls.All methods use one harness at sequence length 2,048 and batch size one; DuoRate read tables are calibrated per benchmark from its own prefill.Source: §4 · pp. 6–7

4.1 MAIN RESULTS

  • At 2.3 bpw, DuoRate averages 73.63, 15.95 points above MoEQuant at 2.5 bpw; at 2.9 bpw, its 76.16 matches NF4 and is within 0.66 points of HIF4.On GSM8K, DuoRate at 2.9 bpw scores 91.00, matching HIF4 at 4.5 bpw and staying within 0.25 points of BF16; at 2.6 bpw, its 89.50 matches or exceeds NF4 and FP4 at 4.5 bpw.Source: §4.1 · pp. 7–8, Figure 3 · p. 7

4.2 WHAT THE ALLOCATION IS WORTH

  • Against the strongest same-rate random mixture, DuoRate improves WikiText-2 perplexity by 1.35% at 2.6 bpw and 1.04% at 2.9 bpw.The corresponding read-rate savings are 0.105 bpw and about 0.07 bpw; the 2.9-bpw crossing is extrapolated. Comparisons with the interpolated uniform-tier null also favor DuoRate, including when tables are re-solved from a disjoint split.Source: §4.2 · p. 8, Figure 4 · p. 8

4.3 CHOOSING THE ALLOCATION OBJECTIVE

  • At matched read rates of 2.6 and 2.9 bpw, the MC-MoE objective has 0.42% and 0.61% higher perplexity, while using 0.29 and 0.43 bpw less stored rate.Neither allocation dominates: frequency weighting favors rarely routed experts in storage, but saves little read rate, so the preferred table depends on which rate is fixed.Source: §4.3 · pp. 8–9

4.4 EFFICIENCY ACCOUNTING

  • The artifacts imply different storage and decode costs: BitsMoE uses 8.58 GiB and 1,699 MB per token, versus DuoRate’s 14.41 GiB and 521–657 MB per token.These figures are derived from storage layout; end-to-end measurement requires a prefix-read kernel.Source: §4.4 · p. 9, Table 2 · p. 9

4.5 WHEN THE READ RATE MATTERS

  • Reading below the stored rate helps most in batch-one long decoding; growing batches make the routing-weighted rate approach the unweighted rate, while short generations are dominated by prefill.A workload change can be handled by adjusting the read budget without re-encoding weights; profiling takes one prefill pass, and solving a read table takes seconds on a CPU.Source: §4.5 · p. 9

5 LIMITATIONS

The evaluation is limited to one MoE model, and the efficiency figures are inferred from storage layout rather than measured with a fused prefix-read kernel.

5 LIMITATIONS

  • Evaluation covers only Qwen3-30B-A3B; other routing schemes and expert counts remain future work.Source: §5 · p. 9
  • Efficiency figures come from storage layout, not end-to-end measurement with a fused prefix-read kernel.Source: §5 · p. 9
  • Prefill runs at read depth rather than stored depth in these measurements, making reported quality a conservative estimate of the two-stage deployment.Source: §5 · p. 9
  • Read tables are calibrated per workload, so deployments with drifting request mixes must schedule recalibration.Recalibration is cheap: one dot product per expert.Source: §5 · p. 9

6 CONCLUSION

The conclusion frames MoE precision as separate resident and decode rates, then presents DuoRate as a single-artifact way to set them independently. On Qwen3-30B-A3B, it reports strong comparisons at lower read rates and recommends reporting both rates for efficiency claims.

6 CONCLUSION

  • A released 2-bit checkpoint shows a 2.95× store–read gap; DuoRate uses one stored tier and workload-specific prefixes to keep a single artifact with two rates.Source: §6 · pp. 9–10
  • On Qwen3-30B-A3B, DuoRate outperforms 2-bit-class methods at 2.3 bpw, matches 4-bit formats at 2.9 bpw, and on long decoding stays within a quarter point of BF16.Source: §6 · p. 10
  • The authors advocate reporting both storage and read rates for MoE checkpoints and identify end-to-end measurement with a fused prefix-read kernel as the next step.Source: §6 · p. 10

SUPPLEMENTARY MATERIAL

The allocator audit distinguishes DuoRate from prior MoE methods that do not formulate stored and decode rates as separate constraints. The algebra shows how routing frequency affects which read budget is selected, while per-expert tier choices follow a frequency-independent trade-off within a domain.

A POSITIONING DUORATE AMONG PER-EXPERT MOE ALLOCATORS

  • No audited published allocator writes stored and read rates as two functionals, and none puts routing frequency in its rate constraint.Some systems vary decode precision through residency, accuracy budgets, or cache policies instead.Source: A POSITIONING DUORATE AMONG PER-EXPERT… · p. 14
  • MC-MoE’s released allocations fix every block’s total bit count, while GEMQ’s Qwen3 allocations show only 0.07–0.21 Spearman correlation between width and decode frequency.GEMQ’s stored and read rates nearly coincide because its distortion weighting does not scale with routing count, not because it constrains read rate.Source: A POSITIONING DUORATE AMONG PER-EXPERT… · p. 14

B FREQUENCY CANCELLATION: ALGEBRA

  • For each expert, the positive routing-frequency factor multiplies both distortion and read-rate terms, so it cancels from the per-expert arg min.Frequency selects which member of the allocation family meets the budget, but does not change that family.Source: B FREQUENCY CANCELLATION: ALGEBRA · p. 15
  • DuoRate’s cancellation is the discrete tier analogue of mixture-source rate–distortion allocation, replacing mixture components and eigenmodes with experts and finite tiers.The correspondence does not depend on Gaussianity or a closed-form distortion.Source: B FREQUENCY CANCELLATION: ALGEBRA · p. 15
  • Promotion thresholds compare distortion reduction with added bit-width, contain no routing frequency, and skip tiers above an expert’s lower convex hull.Where tier families are convex, the Lagrangian solution reaches the global optimum over multiplier-reachable assignments; nonconvex families can leave gains for an exact solver.Source: B FREQUENCY CANCELLATION: ALGEBRA · pp. 15–16
  • Across domains, total expert frequency still cancels, but expert-specific domain mixtures can change the averaged distortion curves and selected tiers.If every expert has the same domain mixture, the single-domain result applies unchanged.Source: B FREQUENCY CANCELLATION: ALGEBRA · p. 15
  • Removing frequency from the distortion while retaining it in the bit-width term can promote never-routed experts to the deepest tier at no charge.The paper’s tie rule guards against this mirror-image failure.Source: B FREQUENCY CANCELLATION: ALGEBRA · p. 15

C PROTOCOL, RATE DEFINITIONS AND SECONDARY SCORES

This supplementary section details the stored prefix stream, rate accounting, and evaluation controls for DuoRate. Paired comparisons favor the 2.9 bpw allocation over the lower uniform tier, while calibration, routing, and measurement analyses qualify how rates and scores should be interpreted.

Stored stream

  • Each expert stores a base code followed by ordered one-bit refinement planes, so tier selection reads a contiguous byte prefix without re-encoding weights.The checkpoint stores uniform p4; decode can read p2, p3, or p4 prefixes, and the common sparse side record is included in every rate and quality number.Source: C PROTOCOL, RATE DEFINITIONS AND… · p. 16, Figure S2 · p. 17

Rate definitions

  • The stored rate is the unweighted mean expert rate, while budgeted read uses calibration prefill frequencies and realized read substitutes decode routing frequencies.The solver meets its target within 0.001 bpw against calibration frequencies; independent routing dumps can produce different realized rates.Source: C PROTOCOL, RATE DEFINITIONS AND… · p. 17

Routing and read rates

  • Experts never reached by calibration prompts account for 0.48% of GSM8K decode traffic and 20% for MMLU, whose prompts are only 36–89 tokens long.Within layers, deeper tiers tend to go to frequently routed experts, while rarely routed experts remain shallow; Figure S5 displays the layer rates.Source: C PROTOCOL, RATE DEFINITIONS AND… · pp. 18, 21, Figure S5 · p. 21

Evaluation protocol

  • Perplexity comparisons use aggregate document-level bootstrap intervals and paired sign tests, while accuracy differences below about one point are treated as ties absent a paired test.The sign test uses 62 documents; accuracy cannot resolve effects around a tenth of a bit at the studied scale.Source: C PROTOCOL, RATE DEFINITIONS AND… · p. 18

Evaluation controls

  • The WikiText-2 interpolated uniform-tier null represents the same format at the mixed table’s rate under continuous precision and no allocation information.On two budgeted cells with real mixtures, synthetic and real controls agree in sign and within 0.11 to 0.14 percentage points.Source: C PROTOCOL, RATE DEFINITIONS AND… · p. 19, Table S5 · p. 19

Paired comparisons

  • At 2.9 bpw, DuoRate wins on all 62 WikiText-2 documents and by 3.36 MMLU-Pro points against uniform p2; against uniform p3, it ties on all three accuracy benchmarks.Uniform p2 reads 0.63 bpw less, while uniform p3 reads 0.37 bpw more; against p3, DuoRate trails by 4.39% in perplexity.Source: C PROTOCOL, RATE DEFINITIONS AND… · p. 19, Table S6 · p. 20

Calibration and allocation checks

  • The stored-p4 cap clips 15 and 29 WikiText-2 experts at 2.6 and 2.9 bpw, respectively, and changes 23 to 37 tier assignments overall.The pooled output profile leaves 87 of 18,432 matrices without measured A; using prompt-only rather than full-sequence routing changes 1.27% of decisions.Source: C PROTOCOL, RATE DEFINITIONS AND… · pp. 20–22

Calibration and evaluation items

  • WikiText-2 activation moments were collected on the scored test documents, so the authors also solve a table using the validation split and report both.For GSM8K and MMLU-Pro, the calibration and evaluation items do not coincide; MMLU has one repeated question among 1,419.Source: C PROTOCOL, RATE DEFINITIONS AND… · p. 20

D AUDIT OF THE EXTERNAL BASELINE

The audit recomputes the released BITSMOE checkpoint’s stored and batch-one read rates, then examines how protocol and accounting choices bound comparisons with the paper’s results. Measurements on earlier stream instances are reported as structural evidence, not interchangeable magnitudes.

D AUDIT OF THE EXTERNAL BASELINE

  • The released expert segment is 2.5425 bpw, not its 2.0 label; independent checks reproduce the reported 8.58 GiB footprint.The audit also finds its released per-projection budget exceeds the stated equation’s allowance.Source: D AUDIT OF THE EXTERNAL BASELINE · p. 22
  • At batch one, BITSMOE stores 2.5425 bpw but reads 7.5008 bpw after never-fetched fields are refunded.The FP16 shared basis is charged to the k=8 selected experts, not all E=128.Source: Figure S6 · p. 22, D AUDIT OF THE EXTERNAL BASELINE · p. 23
  • The shared basis costs 0.3333 bpw in storage versus 5.3333 bpw at batch-one decode (E/k=16), consistent with Zhao et al. (2026)’s per-layer reuse description.The released code computes the bases outside routing, as unconditional dense products.Source: D AUDIT OF THE EXTERNAL BASELINE · p. 23
  • OpenBookQA and WinoGrande align within 0.80 points under coincident protocols, while the HumanEval and GSM8K differences are nonsignificant and opposite in direction.The generative benchmark sizes cannot resolve differences of this magnitude; the runs also use different hardware and dtypes.Source: D AUDIT OF THE EXTERNAL BASELINE · pp. 23–24
  • On common footing, the MMLU quantization gap is 4.4 points here versus 7.9 in BITSMOE’s report, after accounting for few-shot and weighting differences.The comparison also notes that four MMLU subjects have contexts truncated by the 2,048-token window.Source: D AUDIT OF THE EXTERNAL BASELINE · p. 24
  • The GSM8K comparison uses 400 versus 1,319 questions, but unrecorded generation-cap differences prevent a safe correction to the full set.Hardware, dtype, kernels, and harness-version uncertainty further constrain cross-column comparisons.Source: D AUDIT OF THE EXTERNAL BASELINE · p. 24
  • With no allocation, uniform p2 beats released BITSMOE by 2.25 MMLU, 6.43 MMLU-Pro, and 10.00 GSM8K points; this is a format comparison, not an allocation estimate.The audit says the same-format, rate-matched controls are the measure of the allocation itself.Source: D AUDIT OF THE EXTERNAL BASELINE · p. 25

E MEASUREMENTS ON EARLIER INSTANCES OF THE STREAM

E.1 HOLDING ONE FUNCTIONAL AND READING THE OTHER

These earlier-instance experiments contrast storage-only and read-side constraints, then examine storing one artifact for several workloads. Read-side constraints subsume storage constraints under same-domain calibration, while joint storage across workloads can reduce the stored ceiling.

E.1 HOLDING ONE FUNCTIONAL AND READING THE OTHER

  • At 3.60 bpw storage-only allocation correlates more with routing frequency (0.70–0.80) than sensitivity (0.33–0.58); adding the read term shifts these to 0.01–0.13 and 0.71–0.87.Across routed experts, frequency and sensitivity are nearly uncorrelated; under same-domain calibration the read constraint subsumes storage.Source: §E.1 · pp. 25–26

E.2 ONE ARTIFACT FOR SEVERAL WORKLOADS

  • For three workloads at a 3.60 bpw read budget, the union stores 3.749 bpw rather than 6.530; joint storage saves another 0.188 bpw (5.0%) without changing read rates.Perplexity changes by +0.003; pushing storage to 90% of the union moves workload read rates off target.Source: §E.2 · p. 26
  • Truncating the stored ceiling from p5 to p4 to p3 raises perplexity from 8.5796 to 8.6002 to 8.7239.The construction stores each expert at the maximum tier needed across workloads, while each workload reads its own tier.Source: §E.2 · p. 26

E.3 ONLINE RECALIBRATION ACROSS DOMAINS

Online recalibration updates a prose-calibrated allocation using an unlabeled code stream. On the earlier instance, it closes most of the perplexity gap to offline code calibration, with fewer experts changing precision after the first steps.

E.3 ONLINE RECALIBRATION ACROSS DOMAINS

  • On a 150-document code corpus, online recalibration closes 95.3% of the gap to offline code calibration, reaching 4.8320 perplexity versus 4.8245 offline.It starts from prose statistics; Figure S7 depicts precision changes and code-corpus perplexity.Source: Figure S7 · p. 26, §E.3 · p. 26
  • The first three recalibration steps change 3,723, 1,039, and 600 of 6,144 experts, then fewer than 5% change; domain distortion weights have Spearman correlation −0.38.The first step’s size is attributed to the domain shift.Source: §E.3 · p. 26

E.4 THE DISTORTION FUNCTIONAL

This earlier-instance ablation compares distortion functionals across budgets. The decision-aligned criterion performs near the control in reported MMLU results while using less rate at upper budgets, but the authors treat the generative findings as indicative.

E.4 THE DISTORTION FUNCTIONAL

  • At three upper budgets, the decision-aligned criterion stores 0.06–0.16 bpw less than the control while staying within 0.07 MMLU points at three budgets.The squared-error criterion sits below the uninformed control at all four budgets.Source: §E.4 · p. 26, Table S10 · p. 27
  • At 3.200 bpw, the decision-aligned arm is 0.07 points from control (p = 1.00) and 0.91 above the squared-error arm (p = 0.16).The test uses paired per-question logs on MMLU (n = 1,425).Source: §E.4 · p. 27, Table S10 · p. 27
  • Pooled generative results give +2.28 points at the two lower budgets (95% CI [+0.50, +4.11], p = 0.013), but −0.67 at 3.6 bpw (p = 0.56).No single generative benchmark cell is significant, and the ablation predates the later controls, so its positive cells are indicative.Source: §E.4 · p. 27

F.1 A TIER FAMILY THAT WAS NOT ON THE CONVEX HULL

The earlier, separately stored tier family tests allocation heuristics and distortion choices, while the current-stream MC-MoE comparison exposes a storage/read trade-off.

F.1 A TIER FAMILY THAT WAS NOT ON THE CONVEX HULL

  • The earlier family used separately stored formats at 2.6, 3.6, 4.6, and 8.0 bpw, so its measured magnitudes do not transfer to the nested-prefix stream; its 2.6-bpw reference is not an allocation control.That reference scored 47.37 WikiText-2 perplexity, 37.21 MMLU-Pro, and 68.25 GSM8K; the appendix identifies the collapse as a property of a format without a refinement ladder.Source: §F.1 · pp. 27–28
  • At a realized 3.68 bpw, the frequency heuristic scored 78.87 MMLU, 50.43 MMLU-Pro, and 11.06 WikiText-2 perplexity, versus 83.01, 56.36, and 8.50 for the sensitivity allocator at 3.60 bpw.On GSM8K at a realized 3.6 bpw, the heuristic, sensitivity allocator, and uniform format tied at 89.75, 88.75, and 89.00, respectively.Source: §F.1 · p. 27
  • Against a frequency-only control with the same per-layer tier histogram and read rate within 0.03%, sensitivity allocation improved WikiText-2 perplexity by 6.10% at 3.2 bpw and 2.76% at 3.6 bpw.It also gained 2.21 and 1.71 MMLU-Pro points; the control fell below uniform on MMLU-Pro but beat it on WikiText-2 and code perplexity.Source: §F.1 · p. 27
  • At a realized MMLU read rate of 3.000 bpw, the decision-aligned functional scored 55.71 MMLU-Pro accuracy, versus 53.71 for activation-weighted squared error, but the difference was not significant after correcting for four arms.Figure S8 reports the five functionals; their WikiText-2 perplexities ranged from 8.99 to 10.57, with activation-weighted squared error lowest and final-position-only highest.Source: §F.1 · p. 28, Figure S8 · p. 28
  • Decision-aligned weights calibrated on the served domain scored 86.00 GSM8K under flexible extraction and 85.50 strict, versus 81.00 for squared error; cross-domain calibration did not improve the strict score.The cross-domain comparison was 79.50 strict versus 81.25, with p = 0.50; the authors made same-domain calibration of A a requirement.Source: §F.1 · p. 28

F.2 A PUBLISHED ALLOCATOR’S OBJECTIVE ON THE CURRENT STREAM

  • At matched read rates of 2.6 and 2.9 bpw, MC-MoE loses to DuoRate by 0.42% and 0.61% of perplexity, while costing 0.29 and 0.43 bpw less to store.The sign test detects the differences, but the reported intervals include zero; neither allocation dominates across read and storage costs.Source: Figure S9 · p. 28, §F.2 · p. 29
  • For the MC-MoE comparison, the authors set its gate-weight exponent to zero because no routing dump existed for the prose domain, and set the other two exponents to one.MC-MoE’s exponents were not numerically specified or ablated in its own paper.Source: §F.2 · p. 29

F.3 NEGATIVE RESULTS

Several allocation signals and proxy improvements did not yield demonstrated end-to-end gains on the current stream or establish a cross-layer budget result.

F.3 NEGATIVE RESULTS

  • Per-projection units improved the proxy objective by 17% and MMLU by 0.15 points on the earlier tier family, but were not re-measured on the current stream.The reported MMLU change was inside the noise floor.Source: §F.3 · p. 29
  • Seven proxy-objective differences, including a 17% change, a tier-histogram shift, and 0.1 bpw of ledger, failed to appear end to end; no proxy figure is presented as a result.Source: §F.3 · p. 29
  • A preliminary matched-stored-rate probe of layer-level tier assignment scored 76.3 MMLU with distortion-based selection, 78.1 at random, and 79.6 for the shallowest 24 layers, but supports no conclusion.The runs were single and unpaired, used the earlier tier family, and differed from the tables’ working point.Source: §F.3 · p. 29

Related papers

Loading 2610.13328v1…