PaperlayerView exact revision on arXiv

Source-linked AI summary

HodgeCover: Higher-Order Topological Coverage Drives Compression of Sparse Mixture-of-Experts

Tao Zhong, Dongzhe Zheng, Christine Allen-Blanchette

arXiv:2605.13997v1cs.LGcs.AIcs.CL

TL;DR

Learning-free MoE compressors miss higher-order cases where pairwise-compatible experts are jointly non-mergeable. HodgeCover isolates and covers this harmonic obstruction, and its hybrid leads aggressive-compression results across three MoE scales.

  • Problem

    Pairwise expert scores cannot represent irreducible three-expert merge cycles, limiting learning-free compression of sparse MoE layers without retraining.

  • Method

    HodgeCover Hodge-decomposes expert merge barriers and greedily covers harmonic-critical edges and triplet-critical triangles, with a hybrid variant adding weight pruning.

  • Results

    At 66% expert reduction, HodgeCover+Wanda achieves the best perplexity on every model and outperforms STUN+Wanda by up to +12.6 pp downstream accuracy.

  • Takeaways & Limitations

    The harmonic kernel carries mergeability information that pairwise scoring cannot express, and covering it drives the best aggressive-compression frontier across three production MoE scales.

  • Takeaways & Limitations

    HodgeCover remains learning-free, incurs barrier-computation cost, and is evaluated only on language MoE checkpoints.

Abstract

from arXiv · show

Sparse Mixture-of-Experts (MoE) layers route tokens through a handful of experts, and learning-free compression of these layers reduces inference cost without retraining. A subtle obstruction blocks every existing compressor in this family: three experts can each be pairwise compatible yet form an irreducible cycle when merged together, so any score that ranks experts on pairwise signals is structurally blind to which triples are jointly mergeable. We show the obstruction is a precise mathematical object, the harmonic kernel of the simplicial Laplacian on a 2-complex whose vertices are experts, whose edges carry KL merge barriers, and whose faces carry triplet barriers; Hodge-decomposing the edge-barrier signal isolates the kernel exactly. We turn the diagnostic into a selection objective: HodgeCover greedily covers the harmonic-critical edges and triplet-critical triangles, and a hybrid variant of HodgeCover pairs it with off-the-shelf weight pruning on survivors. On three open-weight Sparse MoE backbones under aggressive expert reduction, HodgeCover matches state-of-the-art learning-free baselines on the expert-reduction axis, leads on the aggressive-compression frontier of the hybrid axis, and uniquely balances retained mass across all four Hodge components. These results show that exposing the harmonic kernel of a learned MoE structure changes which compressor wins at the regime that matters most.

1 Introduction

HodgeCover addresses a higher-order mergeability obstruction in sparse MoE compression: pairwise-compatible experts can be jointly non-mergeable, a structure captured by the harmonic component of a simplicial Hodge decomposition. It converts this diagnosis into greedy coverage of harmonic-critical edges and triplet-critical triangles, with HodgeCover+Wanda extending the approach to survivor-weight pruning.

  • Motivation: Sparse MoE layers route each token through only a handful of experts, leaving substantial parameter mass in inactive experts and motivating learning-free compression without retraining.The context identifies Mixtral, OLMoE, Qwen 3.5, and DeepSeek-V3 as production-grade open checkpoints with dozens to hundreds of experts per layer.
  • Problem: Pairwise-compatible experts can form a collectively non-mergeable triple, so compressors based on pairwise rankings cannot detect this higher-order obstruction.The passage contrasts this limitation with REAP, REAM, MC-SMoE, and STUN, which rank or cluster experts using pairwise signals.
  • Method: The method represents experts as vertices, pairwise KL merge-barriers as edges, and triplet barriers as 2-faces in a 2-dimensional simplicial complex.The simplicial Laplacian is used to Hodge-decompose the edge-barrier signal into gradient, curl, and harmonic components.
  • Method: HodgeCover selects k survivor experts by greedy submodular coverage of the top-p% harmonic-critical edges and triplet-critical triangles, using saliency adapted from REAP.Non-survivors are pruned with a router redirect to the closest survivor.
  • Contributions: HodgeCover+Wanda pairs expert selection with unstructured Wanda pruning on survivor weights, extending compression across expert and weight axes.The hybrid variant combines the learning-free HodgeCover objective with unstructured pruning after survivor selection.
  • Contributions: 29 – 62% of per-layer barrier energy is harmonic across three MoE families, verifying that the higher-order component is non-trivial.The contribution passage identifies this harmonic component as the higher-order mergeability obstruction.

2 Related Work

Prior learning-free MoE compression ranks experts using pairwise signals, while weight pruning operates on matrices and misses expert-level redundancy. HodgeCover bridges these limitations by combining higher-order topological expert reduction with unstructured Wanda pruning and relates this approach to broader topological and triplet-aware methods.

  • MoE expert compression: Learning-free MoE pruning and merging ranks experts using pairwise calibration signals, including saliency, routing-cosine clustering, and structured-then-unstructured decisions.Named methods include REAP, REAM, MC-SMoE, HC-SMoE, and STUN.
  • Weight compression: Weight compression spans unstructured pruning, structured pruning, and post-training quantization, but these matrix-level axes cannot exploit redundancy among MoE experts.Examples include Wanda, SparseGPT, LLM-Pruner, ShortGPT, GPTQ, AWQ, SmoothQuant, and OmniQuant.
  • Hybrid compression: HodgeCover+Wanda combines Stage-1 expert reduction using topological signals with Stage-2 unstructured Wanda pruning on survivor weights.The pipeline is learning-free and covers expert-count and weight-level compression axes.
  • Topological and spectral methods: Topological neural-network methods divide between architectural uses of Hodge and simplicial operators and diagnostic uses of Betti numbers and persistence diagrams.The architectural line includes geometric deep learning, simplicial neural networks, and topological GNNs; Hodge theory is traced to Eckmann and later discrete-data work.
  • Triplet and hypergraph methods: Triplet- and hypergraph-aware methods represent higher-order relations through hyperedges, clique-expansion Laplacians, triplet constraints, spectral cuts, and metric-learning margins.These approaches span hypergraph spectral methods, higher-order Cheeger inequalities, motif partitioning, hypergraph convolution and attention, and metric learning.

3 The Mergeability Complex and Its Hodge Decomposition

The mergeability complex models pairwise and triplet merge barriers as signals on a simplicial 2-complex, whose unweighted 1-Hodge Laplacian separates gradient, curl, and harmonic structure. The harmonic component captures barrier energy unexplained by vertex potentials or triangle boundaries and motivates HodgeCover’s coverage constraints.

  • Compression setup: Learning-free MoE compression selects k survivors and redirects removed experts using only a small calibration corpus, without fine-tuning, distillation, or LoRA.The compressed layer minimizes calibration-set KL divergence under a single forward pass over D.
  • Barrier signals: Pairwise barriers measure frequency-weighted two-expert merge costs, while triplet barriers analogously measure joint three-expert merge costs.A fallback to unweighted averaging is used when both experts’ routing rates vanish.
  • Mergeability complex: The mergeability complex represents experts as vertices, pairwise barriers as edge signals, and triplet barriers as triangle signals on a curated simplicial 2-complex.Barriers are signals rather than weights inside the Hodge operator; the complete edge set avoids threshold-induced spectral distortions and detached harmonic cycles.
  • Hodge decomposition: Every edge-barrier signal uniquely decomposes orthogonally into gradient, curl, and harmonic components, with the harmonic subspace dimension equal to the complex’s first Betti number.The unweighted 1-Hodge Laplacian makes the harmonic kernel depend only on the combinatorial topology of the complex.
  • Interpretation: The harmonic energy fraction measures barrier energy that no vertex-potential or triangle-boundary explanation can capture, while gradient and curl encode broad expert mergeability and triangle-level coherence.Pairwise methods implicitly recover only the gradient component, whereas HodgeCover targets the irreducible harmonic structure.
  • Production diagnostics: Both harmonic energy and combinatorial discordance remain non-trivial throughout the evaluated MoE models’ depth, supporting coverage of harmonic-critical edges and triplet-critical triangles.HodgeCover ranks edges by |bharm,e| and triangles by |bijk| for top-p% and top-qT% coverage constraints.

4 Method

HodgeCover is a learning-free expert-selection method that identifies harmonic-critical edges and triplet-critical triangles, then greedily selects k survivors using a monotone submodular coverage objective. HodgeCover+Wanda extends it by applying one-shot Wanda pruning to the selected survivors.

  • HodgeCover selection: HodgeCover selects k of n experts by covering harmonic-critical edges and triplet-critical triangles derived from Hodge-decomposed barrier signals.Edges are ranked by |b_harm,e|, while triangles are ranked by raw |b_ijk|; percentile thresholds p and q_T define the critical sets.
  • Coverage objective: Its non-negative monotone submodular objective combines normalized edge coverage, triangle coverage, and per-expert saliency, enabling standard greedy selection.The edge and triangle terms remain in [0, 1] across layers and models, making λ_e and λ_t layer-agnostic.
  • Coverage objective: Per-expert rankings cannot generally express harmonic-critical incidence because shared critical edges require set-valued coverage over the selected survivor set.This limitation applies to rankings that factor through individual vertex scores, including REAP, REAM centroid selection, and MC-SMoE.
  • Router redirect: Dropped experts are removed and their router logits folded into a single survivor, using a Hodge-weighted barrier for redirect assignment while preserving survivor gates and weights bit-exact.The harmonic weighting penalizes routing through edges carrying their own harmonic mass, avoiding reintroduction of the obstruction targeted by selection.
  • Hybrid compression: HodgeCover+Wanda first drops experts at a fixed rate, then applies one-shot unstructured Wanda pruning to survivor weights until the total target rate, reusing the same calibration corpus without extra forward passes.The expert-selection stage is the contribution, while Wanda supplies residual weight sparsity.

5 Experiments

Experiments across three open-weight sparse MoE backbones show that HodgeCover hybrids achieve strong downstream performance under aggressive expert reduction, while preserving a balanced decomposition across Hodge components. Ablations further indicate that both the triangle term and its soft formulation are necessary.

  • 5 Experiments: Experiments benchmark HodgeCover and learning-free baselines on three open-weight sparse MoE backbones compressed to 33% and 66% of expert count.The models span two expert-count regimes and two scales: OLMoE-1B-7B with 64 experts, and two Qwen models with 256 experts.
  • Downstream results: HodgeCover+Wanda has the best C4 perplexity in every model-rate cell and the best WikiText perplexity in every 66% cell.At 66%, its WikiText margins over STUN+Wanda range from 1.4–3.8 points across the three backbones.
  • Downstream results: At 66% expert reduction, HodgeCover+Wanda leads REAM and MC-SMoE by 6–16 pp DS-Avg on the four Qwen cells while matching REAP within ±0.3 pp.On OLMoE, it retains 48.6% DS-Avg and 21.2 C4 perplexity, recovering performance where every 66% expert-count cell loses 23–25 pp DS-Avg and exceeds 100 C4 perplexity.
  • Hodge-component retention: HodgeCover stays close to the backbone’s pre-compression decomposition across harmonic, gradient, curl, and triplet-barrier components, unlike baselines with sharply skewed retention.This balanced retention corresponds to a 6–16 pp DS-Avg advantage over Greedy-Barrier and MC-SMoE on the two Qwen scales.
  • Ablations: At 66%, HodgeCover gains +5.7 pp DS-Avg over Triplet-Penalty at virtually identical C4 perplexity, while removing triangles costs −11.5 pp and a binary triangle veto costs −30.5 pp.These ablations identify both the triangle term and its soft formulation as necessary.

6 Conclusion … A.5 Explicit projection formulas

HodgeCover exposes higher-order MoE mergeability through the harmonic component of a simplicial complex, then covers harmonic-critical edges and triplet-critical triangles for aggressive compression. The appendices formalize the 2-complex, Hodge operators, decomposition, and efficient projection implementation, while noting learning-free and evaluation limitations.

  • 6 Conclusion: HodgeCover reveals irreducible three-expert cycles invisible to pairwise merge scores and uses harmonic-critical edges and triplet-critical triangles to drive aggressive compression.The simplicial mergeability complex makes higher-order incompatibility visible through its harmonic component.
  • 6 Conclusion: The method remains learning-free; recovering the last few perplexity points typically requires fine-tuning or knowledge distillation, and evaluation covers only language MoE.Plan-time barrier computation on a 35B-parameter MoE is a one-shot offline cost amortized across compression rates because the complex is cached.
  • A Hodge Decomposition Primer and Implementation Details: The appendix develops the abstract simplicial-complex background, boundary operators, 1-Hodge Laplacian, Hodge decomposition, and Moore-Penrose projection formulas.It also documents implementation details deferred from Section 3.
  • A.1 Abstract simplicial complexes: The complex uses vertices, complete pairwise edges, and curated triangles, with simplices oriented canonically by a fixed total ordering.The paper works with q ∈ {0, 1, 2}: vertices, edges, and triangles, using unweighted Euclidean chain spaces.
  • A.2 Boundary operators and the chain identity: Boundary maps represent oriented edge-to-vertex and triangle-to-edge incidence, and satisfy the chain identity ∂1 ◦ ∂2 = 0.This boundary relation distinguishes the simplicial 2-complex from a hypergraph and enables the Hodge decomposition.
  • A.3 The combinatorial 1-Hodge Laplacian and its kernel: The 1-Hodge Laplacian combines lower edge adjacency through ∂1 with upper edge co-face coupling through the chosen triangle set T.Its kernel dimension is the first Betti number β1(K), while the weighted variant preserves the kernel topology but changes eigenvectors and projectors.
  • A.3 The combinatorial 1-Hodge Laplacian and its kernel: For Qwen3.5-35B and Qwen3.5-122B, β1(K) = 31,885 at every layer and rank ∂2 = 500, making the curl subspace exactly full-rank.For OLMoE, β1(K) ranges from 1453 to 1455 across 16 layers, with rank ∂2 from 498 to 500.
  • A.4 The discrete Hodge decomposition: Every edge signal decomposes uniquely and orthogonally as b = bgrad + bcurl + bharm across gradient, curl, and harmonic subspaces.The harmonic subspace is ker(L1), completing the direct-sum decomposition of C1(K).

A.6 Harmonic energy as irreducible mergeability residual … A.11 Calibration corpus

The appendix formalizes harmonic energy as the irreducible part of edge-barrier signals, then specifies conditional loss interpretation, complex construction, frequency-weighted merging, and a fixed calibration corpus. It also explains why thresholding and triangle-set choices materially affect the Hodge diagnostic.

  • A.6 Harmonic energy as irreducible mergeability residual: The harmonic residual is the minimum squared error remaining after explaining edge barriers with vertex potentials and triangle-boundary corrections.The unique best lower-order explanation is m⋆ = bgrad + bcurl, while the residual energy equals ∥bharm∥2.
  • A.6 Harmonic energy as irreducible mergeability residual: The normalized harmonic energy measures the edge-barrier energy that lower-order explanations cannot capture, without itself providing an unconditional calibration-KL lower bound.That loss connection requires an explicit edge-exposure model and is not assumed by HodgeCover.
  • A.7 Conditional link to compression loss under edge-exposure linearization: Under edge-exposure linearization, harmonic exposure determines the harmonic-blind prediction and pairwise ranking errors, up to the stated linearization residual.No barrier component aligned with blo contributes to this error, but the approximation hypothesis is interpretive rather than an HodgeCover assumption.
  • A.8 Effect of edge thresholding: Thresholding edges can change graph connectivity, curl structure, and Betti number non-monotonically, making harmonic energy depend strongly on the threshold.The harmonic projector can detach a cycle from ker(L1) and reattach it as a boundary after a small threshold change.
  • A.9 Triangle-set construction and sensitivity: Triangle candidates are median-threshold 3-cliques, capped at |T|max = 500 with fixed seed 42, and the Hodge complex uses a Betti-maximizing filtration threshold.Across the studied layers, the selected threshold retains the complete edge set and the full candidate triangle set.
  • A.9 Triangle-set construction and sensitivity: The candidate triangle set controls the curl subspace and harmonic dimension, while exhaustive triangle enumeration is computationally infeasible.With no triangles, harmonic energy contains all non-gradient signal; a curated subset provides an intermediate regime.
  • A.10 Frequency-weighted merge and the all-zero-frequency guard: Frequency-weighted expert merges use routing frequencies, falling back to an unweighted average when total frequency is effectively zero.The implementation uses ε = 10−12 to prevent 0/0 ratios and keep barrier matrices NaN-free on high-expert-count Qwen layers.
  • A.11 Calibration corpus: All methods compute barriers on the same 2,048-token C4 calibration corpus, sampled with random seed 42 for direct baseline comparability.The corpus is drawn from the C4 training split and is fixed across HodgeCover and the listed comparison methods.

A.12 Per-layer ranges and gradient/curl companion curves · B Algorithmic Details for HodgeCover and HodgeCover+Wanda · B.1 HodgeCover pseudocode

The appendix reports layerwise Hodge diagnostics and specifies HodgeCover’s per-layer greedy selection procedure. Harmonic structure is depth-stable, while discordance varies by model depth and curl contributes little additional signal at |T| = 500.

  • A.12 Per-layer ranges and gradient/curl companion curves: ρharm spans [0.289, 0.491] across OLMoE’s layers, while δ spans [0.596, 1.000], reaching δ = 1.000 on layer 1.The layer-1 value means every sampled triangle is discordant under the 1.2× margin.
  • A.12 Per-layer ranges and gradient/curl companion curves: ρharm is depth-stable across models, whereas δ tapers from high early-layer values to low late-layer values on Qwen and remains uniformly high on OLMoE.The discordance fraction therefore exposes where higher-order obstructions concentrate by depth.
  • A.12 Per-layer ranges and gradient/curl companion curves: ρcurl is 6.3 × 10−5–1.5 × 10−3 on Qwen3.5-35B, 9.8 × 10−5–1.5 × 10−3 on Qwen3.5-122B, and 0.025–0.049 on OLMoE at |T| = 500.Hodge orthogonality gives ρgrad + ρcurl + ρharm = 1 where b(ℓ) ≠ 0; curl is essentially zero on Qwen and around 3% on OLMoE.
  • A.12 Per-layer ranges and gradient/curl companion curves: The diagnostics define δ using a 1.2× margin, counting triangle {i, j, k} as discordant when bijk > 1.2 · max(bij, bik, bjk).The threshold is a representative pipeline value rather than the result of a sweep.
  • B.1 HodgeCover pseudocode: HodgeCover greedily adds the candidate expert with maximum marginal gain until |S| = k, returning survivors S⋆ and redirecting each removed expert to arg minj∈S⋆ bij.The algorithm’s inputs include saliency, protected experts, and hyperparameters p, qT, λe, λt, and α.
  • B.1 HodgeCover pseudocode: The triangle set T is supplied by Stage A’s 3-cliques of the median-pairwise-barrier subgraph, capped at |T| = 500 by fixed-seed uniform-random subsampling when necessary.The threshold τ⋆ is selected by the layer-specific Betti-maximizing filtration sweep in Stage B.
  • B.1 HodgeCover pseudocode: The marginal-gain loop is monotone submodular greedy maximization with a (1 − 1/e) approximation guarantee and O(k n (∆E + ∆T)) marginal-gain evaluation cost.If E⋆ or T⋆ is empty, its normalized coverage gain is set to zero; vanishing harmonic structure reduces the method to saliency-ranked greedy plus triplet-only coverage.

B.2 Proof of Proposition 2 and an inexpressibility example … C.5 Triangle-set construction

The appendices establish HodgeCover’s submodular greedy guarantee and show why vertex-only selectors cannot express instance-dependent harmonic coverage. They also document implementation conventions, computational costs, baseline choices, and analytical limitations of the hyperparameters and triangle-set construction.

  • B.2 Proof of Proposition 2 and an inexpressibility example: HodgeCover’s objective is non-negative monotone submodular, so greedy selection under |S| = k achieves Φ(S⋆) ≥ (1 − 1/e) max|S|=k Φ(S).The guarantee follows from additive saliency plus normalized edge- and triangle-coverage functions; the factor (1 − 1/e) ≈ 0.632 is a worst-case bound, not claimed tight for these instances.
  • B.2 Proof of Proposition 2 and an inexpressibility example: Individualistic vertex-score selectors can return the same pair across symmetric K4 instances, while harmonic projection shifts the critical-edge set and changes coverage.REAP, REAM centroid selection, MC-SMoE, and published STUN fit the vertex-score template; HodgeCover instead uses joint critical-edge incidence.
  • B.3 Cross-layer expert-budget allocation: HodgeCover and REAP-family baselines use uniform per-layer survivor allocation, whereas MC-SMoE and STUN+Wanda retain native global allocation rules.The alternative compressibility-weighted allocator was implemented but excluded from main-table results to avoid conflating survivor selection with budget allocation.
  • B.4 HodgeCover+Wanda Stage 2: Stage 2 of HodgeCover+Wanda applies published row-wise Wanda independently to surviving experts and shared MoE linear layers, reusing Stage 1 activations without another forward pass.The fixed Stage-1 drop rate is r1 = 0.20, while Stage-2 sparsity matches total compression targets rtot ∈ {0.33, 0.66}; matched hybrids use the same schedule.
  • B.5 Plan-time complexity and wall-clock cost: Plan-time cost scales as L · n^2 in barrier sweeps, with Hodge projection under two seconds per layer at n = 256 and greedy selection well under one second.The pairwise-barrier sweep dominates offline cost; the triplet sweep is capped at |T| = 500 and the projection remains a sparse linear-algebra operation.
  • B.6 Baseline-axis comparison grid; B.7 Baseline implementation notes: External baselines that perturb survivor weights use learning-free per-pair heuristics, whereas HodgeCover keeps survivors bit-exact to avoid merging experts carrying irreducible harmonic mass.MC-SMoE is reported without its published 20,000-step knowledge-distillation pass because that pass overfits and regresses quality at the paper’s 2,048-token calibration scale.
  • C Hyperparameter and Design-Choice Sensitivity; C.1 Defaults and how they were set; C.3 Coverage weights λe, λt; C.4 Harmonic-redirect strength α: The defaults are p = qT = 20%, λe = 1.0, λt = 0.5, α = 3.0, and |T| ≤ 500, selected qualitatively on one validation layer and reused across experiments.The coverage weights have uniform interpretation because normalized marginal gains lie in [0, 1], while formal multi-scale weight sensitivity sweeps were left for future work.
  • C.2 Critical-simplex fractions p, qT; C.5 Triangle-set construction: Critical-simplex fractions have analytical boundary behavior: large p, qT saturate coverage, small values reduce it to saliency-plus-noise, and the default yields |E⋆| ∈ [400, 6500] without empirical sensitivity bands.Triangle construction compares pairwise-only, all-triples, and curated regimes; all reported results use the curated |T| ≤ 500 cap, and no paired multi-cap ablation was run.

C.6 Merge metric

The paper defines merge barriers using KL divergence between original and merged layer output distributions on a calibration corpus. This choice matches the categorical language-modeling output and the KL-based calibration objective.

  • C.6 Merge metric: Merge barriers b_ij and b_ijk are computed as KL divergence between original-layer and merged-layer output distributions on calibration corpus D.The computation is specified in Appendix A.10.
  • C.6 Merge metric: KL is used because language-model outputs are categorical distributions over the vocabulary, matching the modeling regime.
  • C.6 Merge metric: The metric also matches the calibration objective in Equation 1, which is itself a KL divergence.

D Experimental setup details … E Additional Results

The appendix details the evaluation setup for learning-free compression across three open-weight sparse MoE backbones, fixed calibration and evaluation protocols, and additional analyses spanning allocators, hybrids, Hodge components, and task-level ablations.

  • D Experimental setup details: The appendix supports Section 5.1 and documents the experimental setup and additional results.
  • D.1 Models and checkpoints: Three canonical, non-fine-tuned open-weight sparse MoE backbones cover small-model and high-cardinality regimes.OLMoE-1B-7B uses 16 MoE layers, 64 experts per layer, and top-2 routing; the two Qwen models use 40 or 48 layers, 256 experts per layer, and top-8 routing.
  • D.2 Compression rates and the per-layer allocator: 33% and 66% expert-removal rates are selected to straddle the easy/hard frontier from prior MoE-compression literature.Per-layer survivor counts are produced by each baseline’s source-paper allocator.
  • D.3 Calibration and evaluation: Calibration uses 2,048 randomly sampled C4 train-split tokens at seed 42, while evaluation includes WikiText-103 perplexity, held-out C4 perplexity, and nine downstream tasks.Calibration tokens are fixed across methods, concatenated to the model’s training context length, and truncated to the calibration budget.
  • D.4 Hardware and reproducibility: Primary calibration, planning, and topology audits run on a workstation with 2 NVIDIA RTX PRO 6000 Blackwell GPUs, 252 GB RAM, and 64 CPU cores.OLMoE Round-N benchmarks used an A6000 workstation, while Qwen ablations and topology audits used H200 nodes; throughput uses PyTorch 2.x and Hugging Face transformers.
  • E Additional Results: Additional results cover per-method allocators, four HodgeCover ablations, matched-control hybrid comparisons, and per-rate Pareto analysis.
  • E Additional Results: The appendix also reports per-layer retained-mass trajectories for harmonic, gradient, curl, and triplet-barrier components, plus per-task ablation breakdowns across all three backbones.

E.1 Per-method allocator and ablation construction · E.2 Full hybrid axis with matched-control baselines · E.3 Mechanism: H/G/C/T retained mass on OLMoE and Qwen 3.5-35B-A3B

The appendix specifies each method’s survivor allocator and isolates HodgeCover’s triangle and Hodge-decomposition contributions through controlled ablations. Matched hybrid results favor HodgeCover+Wanda at higher reduction rates, while retained-mass analyses show distinct trade-offs across gradient, harmonic, curl, and triplet components.

  • E.1 Per-method allocator and ablation construction: Baseline allocators differ: REAP uniformly drops low-output-saliency experts, REAM clusters to fixed survivor counts, and MC-SMoE merges low-frequency experts into router-similar dominants.These implementations follow the respective released source-paper procedures.
  • E.1 Per-method allocator and ablation construction: STUN+Wanda combines STUN behavioral-similarity selection with a Stage-2 unstructured Wanda sweep, whereas HodgeCover uses REAP’s uniform per-layer allocator.The four HodgeCover ablations separately vary triangle coverage and Hodge decomposition, with alternative merge procedures using frequency-weighted aggregation.
  • E.1 Per-method allocator and ablation construction: HodgeCover is the triangle-on, Hodge-on configuration; Hodge No-Triangle removes only triangle coverage, while Greedy-Barrier, Triplet-Penalty soft, and Triplet-Hypergraph hard remove or replace Hodge-based selection.Greedy-Barrier sorts pairwise barriers; the soft variant adds triplet penalties, and the hard variant vetoes unions containing high-barrier triples.
  • E.2 Full hybrid axis with matched-control baselines: At 66% on the two Qwen scales, HodgeCover+W opens a 0.3–0.8 pp DS-Avg gap over REAP+W and REAM+W and a 5–13 pp gap over STUN+W.At 33%, matched-control hybrids cluster within 0.8 pp DS-Avg, with HodgeCover+W and REAP+W trading task-level leads.
  • E.2 Full hybrid axis with matched-control baselines: 48.6% DS-Avg is achieved by HodgeCover+W on OLMoE-1B-7B at 66%, compared with 45.1% for STUN+W, 43.3% for REAP+W, and 40.8% for REAM+W.The four hybrids span an 8 pp DS-Avg range on this cell.
  • E.3 Mechanism: H/G/C/T retained mass on OLMoE and Qwen 3.5-35B-A3B: Retained mass evaluates survivor coverage of the original pre-compression Hodge decomposition, averaging per-layer edge and triangle support fractions across MoE layers.The metrics measure how much original structural signal each survivor set covers without re-decomposing the compressed sub-MoE.
  • E.3 Mechanism: H/G/C/T retained mass on OLMoE and Qwen 3.5-35B-A3B: Greedy-Barrier and MC-SMoE retain more harmonic and gradient mass than HodgeCover at the cost of curl and triplet-barrier mass, while Random sweeps reverse this trade-off.REAP and REAM deviate only weakly from HodgeCover on every component and share identical pure-axis survivor selection.

E.4 Full ablation breakdown

The full ablation breakdown extends the Qwen 3.5-35B-A3B analysis across three backbones and two compression rates. HodgeCover’s advantage is strongest on Qwen 3.5-122B-A10B at 66%, while OLMoE forms a harder stress regime.

  • Full ablation breakdown: On OLMoE-1B-7B at 33%, HodgeCover and Triplet-Penalty are within 1 pp DS-Avg, while at 66% every expert-count method collapses toward the 33–36% DS-Avg band.This stress regime makes the topological objective harder to distinguish from the matched soft-penalty baseline because methods remain far below the uncompressed reference.
  • Full ablation breakdown: At 66%, Table 5 compares macro retained mass across harmonic, gradient, curl, and raw triplet-barrier components for HodgeCover and baselines.HodgeCover, REAP, and REAM span all four components, whereas Greedy-Barrier, MC-SMoE, and Random each maximize a strict subset.
  • Full ablation breakdown: Table 6 reports per-task ablations across all three backbones at 33% and 66%, with unweighted nine-task DS-Avg as the aggregate metric.Triplet-Penalty (soft) was not run on Qwen 3.5-122B-A10B pre-submission.
  • Full ablation breakdown: +11.4 pp DS-Avg is HodgeCover’s gain over Hodge No-Triangle on Qwen 3.5-122B-A10B at 66%.The same comparison reports +8.9 pp over Greedy-Barrier and +32.2 pp over Triplet-Hypergraph.

E.5 Systems characterization

Table 7 characterizes HodgeCover hybrids by planning overhead, throughput, routing entropy, and dead-expert ratio. HodgeCover’s one-time complex construction is cached across compression rates, while HodgeCover+Wanda closely tracks REAP+Wanda in throughput but has higher dead-expert ratios than REAM+Wanda on Qwen.

  • Systems characterization: Table 7 measures plan-time at 33% compression and throughput, normalized routing entropy, and dead-expert ratio at 66%.Subsequent compression rates reuse the cached simplicial complex and omit a separate 66% plan-time column.
  • Planning overhead: HodgeCover’s one-time simplicial-complex construction, barrier sweep, and Hodge decomposition are cached per backbone across compression rates.Only survivor selection and redirection add a small per-rate component, together within single-digit seconds.
  • Throughput and routing: HodgeCover+Wanda matches REAP+Wanda’s tokens-per-second within 6% on every backbone.The largest gap is 370 tok/s, 6.0% relative, on OLMoE-1B-7B.
  • Dead-expert ratio: REAM+Wanda has lower dead-expert ratios than HodgeCover+Wanda by 1–3 pp absolute on the two Qwen scales.The reported pairs are 35.1% vs. 37.3% on Qwen 3.5-35B-A3B and 27.5% vs. 29.0% on Qwen 3.5-122B-A10B.
  • Dead-expert ratio: On the same Qwen backbones, REAM+Wanda loses 0.3–0.8 pp DS-Avg to HodgeCover+Wanda despite its lower dead-expert ratios.The paper therefore treats dead-expert ratio as a disadvantage for HodgeCover+Wanda but not as downstream-correlated in these data.
Loading 2605.13997v1…