PaperlayerView exact revision on arXiv

Source-linked AI summary

Relax Within, Balance Across: Geometry-Guided Load Balancing for Vision-Language Mixture-of-Experts

Ziang Wu, Peng Jin, Qishen Yin, Munan Ning, Hao Li, Peizhen Zhang, Li Yuan

arXiv:2608.00574v1cs.CVcs.AI

TL;DR

Vision-language MoE balancing can hide opposing image and text load errors at one token mix, while changing resolutions, tiling, or prompts shifts that mix. ReBA separates modality objectives and weights each image equally, lowering load across reported inputs and tested shifts while maintaining comparable mean task accuracy.

  • Problem

    Std-Aux balances one mixed image–text load, allowing opposing conditional errors to cancel near one token mix while loads become imbalanced as composition changes.

  • Method

    ReBA balances image and text separately and aggregates one equal-weight routing profile per image, following measured modality and image boundaries.

  • Results

    Across four split backbones, ReBA lowers benchmark-input load on every reported task and reduces average-over-range and worst physical load while mean task accuracy remains comparable to Std-Aux.

  • Takeaways & Limitations

    ReBA improves load balance across the tested benchmark inputs and resolution, tiling, and modality-composition shifts without reducing comparable mean task accuracy.

  • Takeaways & Limitations

    The fixed-profile load law isolates token-ratio effects, but physical preprocessing can change conditional load profiles and outweigh those effects.

Abstract

from arXiv · show

Vision-language MoE batches contain different numbers of image and text tokens. Image resolution, image count, tiling, and prompt length all change this token mix. We call the standard token-level Switch auxiliary loss Std-Aux. Std-Aux balances only the mixed load, so large image and text load errors can cancel at one mix. On our main model, the same trained router shows more than a fivefold change in load imbalance across image resolutions. We hold the image and text load profiles fixed and derive the exact load curve as the token mix varies. The image-text load gap controls sensitivity to the token mix. Physical preprocessing can also change the conditional profiles. The fixed-profile law excludes such changes. To design a remedy, we examine the router input structure. Image and text occupy distinct regions, while visual tokens group strongly by source image. The modality boundary motivates separate image and text terms. The image boundary motivates one equal-weight routing instance per image. ReBA, or Relax Within, Balance Across, implements both choices. Across four split backbones, ReBA lowers load on every reported benchmark input while keeping mean task accuracy comparable to Std-Aux. ReBA also lowers average load over the tested range and worst physical load under resolution and tiling shifts. Code is available at https://github.com/ZiangWu-77/ReBA.

1 Introduction

This section shows that vision-language MoE load balancing must remain stable across changing image–text token mixes, because mixed-load cancellation can fail as composition changes. It derives a fixed-profile load law and introduces ReBA, which uses modality-separated terms and equal-weight per-image routing instances.

  • Motivation: Vision-language inference changes the image–text token mix through dynamic resolution, image count, tiling, and prompt length.
  • Failure mode: Std-Aux can balance one composition yet become imbalanced across compositions because image- and text-load errors cancel only near particular token mixes.When the mix changes, the errors receive different weights, breaking cancellation and raising mixed load.
  • Composition-shift law: The fixed-profile law separates best load from token-mix sensitivity, with the image–text load gap controlling curve steepness and low-load-region width.A large gap produces a steep curve and narrow low-load region; a small gap produces a flatter curve and wider region.
  • Geometry-guided ReBA: ReBA responds to router geometry by using separate image and text terms and one equal-weight routing instance per source image.Distinct modality regions motivate separate terms, while strong within-image visual-token grouping motivates per-image instances.
  • Results: ReBA lowers average RMS-CV across tested ratios, worst physical load under resolution and tiling shifts, and benchmark-input mean layer CV on every reported task and split backbone.Mean task accuracy remains comparable to Std-Aux.

2 Background and Related Work

Sparse MoE layers replace dense FFNs with routed parallel experts, using token-level Switch auxiliary balancing for efficient expert-parallel execution. In vision-language settings, this mixed-load objective can hide modality-specific imbalance, motivating ReBA’s conditional routing approach.

  • Sparse MoE layers and load balancing: Sparse MoE layers replace a dense FFN with N parallel experts and route each token using router logits Wr x_t.
  • Sparse MoE layers and load balancing: Std-Aux balances realized expert assignments and soft gate mass, but token-averaged statistics omit modality identity and sample identity.
  • Sparse MoE layers and load balancing: Image and text can remain individually imbalanced because their errors cancel in the mixed load constrained by L_aux.
  • Related work: Prior work studies routing, dense-to-MoE conversion, multimodal MoEs, and expert-parallel inference; ReBA instead changes conditional routing profiles during training.

3 Problem and Diagnosis

Std-Aux can hide large, opposing image and text load errors because it balances only their mixed token load, causing sensitivity to changing image-token ratios. Router geometry motivates ReBA’s separate modality terms and per-image visual routing instances, while the fixed-profile law links conditional load gaps to ratio sensitivity.

  • 3.1 Ratio shift breaks Std-Aux balance: Std-Aux RMS-CV changes by more than fivefold across five resolution settings despite a fixed router and converged auxiliary loss.The resolution sweep changes the image-token fraction while keeping the router checkpoint fixed, revealing a checkpoint-specific balance point rather than uniform balance.
  • 3.2 Hidden cancellation: Std-Aux aggregates all tokens, allowing opposing image and text loads to cancel while the mixed load appears balanced.The standard loss observes one mixed load rather than separate modality loads, so hidden conditional imbalance can remain large.
  • 3.2 Hidden cancellation: Under Std-Aux, mean layer Pearson correlation is −0.949 and G1 = 0.798, versus overall mean layer CV of 0.273.These values show large, opposing conditional loads even when aggregate imbalance is relatively low.
  • 3.3 Observed geometry: Image and text occupy separate router-input regions, while visual tokens form tight same-image routing blocs, motivating separate modality terms and one routing instance per image.In native routers, 52% and 61% of experts receive more than twice as much dispatch from one modality as the other; text remains pooled because its cross-sample gap is roughly twice as large.
  • 3.4 Ratio-sensitive load: The fixed-profile load curve is a parabola whose curvature rises with the image–text conditional load gap, while ReBA reduces that gap and curvature.Expected squared load also grows with ratio variance and when the deployment mean moves away from the checkpoint’s best ratio.

4 ReBA

ReBA separates image and text balancing objectives and treats each image as one visual routing instance. It preserves observed modality composition while changing auxiliary-statistic averaging within each modality, preventing token-count differences from determining optimization weight.

  • Separate modality objectives: ReBA computes separate nonnegative image and text terms, so imbalance in one modality cannot hide imbalance in the other.A small ReBA loss requires both modality loads to approach balance under the aligned hard–soft surrogate.
  • Image-level instances: Each image forms one routing instance containing all its visual tokens, while different images remain separate.Image averaging limits repeated influence from correlated patches and preserves cross-image differences.
  • Instance-level averaging: Equal-instance averaging gives every routing instance the same optimization weight regardless of token count.The soft mean supplies router gradients, while the hard mean measures realized instance-level load.
  • Physical-load preservation: ReBA preserves observed image–text token fractions while changing optimization weight within each modality, without changing physical token counts.Equal weighting applies only to the auxiliary objective; visual tokens still enter routed experts and contribute to image profiles.
  • Implementation: ReBA adds no router or expert parameters and changes only the groups over which standard Switch-style hard–soft statistics are averaged.The extra aggregation costs O(|M|N) per layer.

5 Experiments

Experiments show that ReBA consistently reduces load imbalance across benchmarks, modalities, token mixes, and physical preprocessing shifts while preserving comparable task quality. Ablations attribute the gains to combining separate modality terms with image-level routing instances.

  • Benchmark results: ReBA lowers mean layer CV on every benchmark and backbone, while mean accuracy differs from Std-Aux by +0.3, +0.2, −0.2, and +0.6 points.The four backbones are Split-Qwen3VL-4B, Split-Qwen2.5VL-3B, Split-Qwen2VL-7B, and Split-InternVL3-8B.
  • Conditional load: ReBA changes mean image–text correlation from −0.949 to −0.166 and mean ℓ1 gap from 0.798 to 0.137, reducing cancellation between conditional loads.The lower mixed load therefore does not come from a new image–text cancellation.
  • Image and text routing: Within-image CV falls from 0.48 to 0.21, while image and text between-profile shares rise from 0.19 to 0.79 and 0.30 to 0.87.Image profiles become flatter within each image and more varied across images; the text increase is a learned outcome rather than a direct instance constraint.
  • Token-mix stability: ReBA lowers fixed-profile curvature κ by 96.3% and remains low across the tested ratio range, unlike Std-Aux’s narrow minimum and Coupled-ImgInst’s steep curve.Average-over-range and worst RMS-CV summarize the controlled curve.
  • Physical shifts: ReBA stays below Coupled-ImgInst at all five resolution settings, below Std-Aux at four, and below Std-Aux at every InternVL tile count.It is lowest from three to twelve tiles in the InternVL tiling sweep.
  • Ablations: Ablations show that image instances alone leave cancellation, separate terms alone reach 0.154 overall CV, and full ReBA reaches 0.077 versus Std-Aux’s 0.273.Pooled-text ReBA also outperforms ReBA-TextInst, reaching 0.077 versus 0.098 overall CV.

6 Conclusion … A.2 Dense to Split MoE Construction

ReBA addresses composition-sensitive load imbalance by balancing image and text separately and aggregating image routing per source image, while preserving comparable task quality across split backbones. The appendices specify its implementation and analysis, report workload-shift and diagnostic results, and describe dense-to-split MoE construction that preserves the dense FFN parameter set.

  • 6 Conclusion: ReBA lowers benchmark-input load across four split backbones while maintaining comparable mean task quality, average-over-range load, and worst physical load across tested shifts.The fixed-profile law isolates token-ratio changes.
  • Appendix: The appendix covers ReBA implementation, composition-shift analysis, coefficient and checkpoint studies, expert compute estimates, workload shifts, and native MoE diagnostics.
  • A.1 ReBA Algorithm: ReBA converts token routing into separate image and text objectives using within-instance routing profiles and modality-specific loss terms.The formulation defines N routed experts, top-k selection, token-indexed router inputs, and routing instances.
  • A.1 ReBA Algorithm: Each image forms one routing instance, while all text tokens form one pooled text instance.Image statistics are averaged within each image and then across images; text statistics remain pooled.
  • A.1 ReBA Algorithm: The hard profile measures realized expert use, whereas the soft profile supplies router gradients and penalizes experts jointly high in hard load and soft probability.
  • A.1 ReBA Algorithm: ReBA removes token-count-based image length weighting from the auxiliary objective by averaging image profiles equally.Standard token averaging gives longer images more optimization weight; computing instance identifiers and segment means costs O(T + MN) per layer.
  • A.2 Dense to Split MoE Construction: The dense-to-split construction partitions intermediate neurons across disjoint experts, matching gate, up, and down parameters so summing active expert outputs recovers the dense FFN output.
  • A.2 Dense to Split MoE Construction: The split experts jointly preserve the dense model’s FFN parameters apart from the small N × H router, while sparse routing evaluates only selected experts.The router is initialized at zero, and uniform initial routing gives every expert gradient; with four experts and top-2 routing, each token activates half the original intermediate neurons.

A.3 Training Setup

The study compares four split backbones under matched training settings, changing only the balancing objective among No-Aux, Std-Aux, and ReBA. All models use a common one-epoch supervised fine-tuning setup, with routing analyses and downstream evaluations conducted under fixed protocols.

  • Matched comparison settings: Only the balancing objective changes across No-Aux, Std-Aux, and ReBA, which share the same initial split checkpoint and task training setup.The vision encoder is frozen, while the language model, router, and unfrozen multimodal modules are trained.
  • Training configuration: One epoch of supervised fine tuning on Cambrian-737K uses bfloat16, FlashAttention, a cosine schedule, learning rate 10−5, warmup ratio 0.03, and maximum sequence length 4096.Primary 4B runs use full parameter training, per-device batch four, 16 accumulation steps, recorded global batch 512, NVIDIA H20 GPUs, ZeRO-1, and gradient checkpointing.
  • Analysis protocol: Final routing analyses use the last checkpoint after one epoch, while coefficient studies evaluate λaux ∈ {0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.1}.The fixed routing probe does not choose a checkpoint.
  • Evaluation protocol: Downstream evaluation uses VLMEvalKit with rule based or exact match scoring and no GPT judge, using the same preprocessing and evaluation version within each backbone comparison.The main paper compares four split backbones under matched training settings.

A.4 Routing and Evaluation Protocol · A.5 Result Provenance · B Mathematical Details

The paper evaluates routing balance using dispatched-token load metrics, distinguishes benchmark, diagnostic, and physical-sweep provenance, and formalizes composition-sensitive measures such as RMS CV, AUC, conditional-profile gaps, and fixed-profile curvature. These definitions establish how load imbalance, workload sensitivity, and routing structure are quantified across experiments.

  • A.4 Routing and Evaluation Protocol: Every load result uses true top k dispatched counts, with lower layer CV indicating better expert load balance.Layer CV is zero under uniform routing and increases as expert load departs from uniform.
  • A.4 Routing and Evaluation Protocol: Benchmark tables use MeanCV, while composition experiments use RMS CV so severely imbalanced layers receive greater weight.MeanCV weights layers equally; RMS CV emphasizes layers with large CV.
  • A.4 Routing and Evaluation Protocol: Confidence intervals use 1,000 paired bootstrap resamples, while request-tail metrics compute per-request RMS CV before taking percentiles.Each bootstrap resample recomputes profiles and load metrics from dispatched counts.
  • B Mathematical Details: Normalized AUC reports average RMS CV over a stated image-token-fraction interval, with controlled, physical, and InternVL variants using specified measured-fraction domains.AUC uses uniform interval weighting rather than deployment request-frequency weighting, and lower values are better.
  • B Mathematical Details: Conditional-profile metrics separate image and text routing profiles through G1 or G2, while fixed-profile curvature κ measures how rapidly squared RMS CV changes with image-token fraction.G1 and G2 should not be compared numerically: split-model results report G1, whereas native tables report G2; κ is defined only for fixed conditional profiles.
  • A.4 Routing and Evaluation Protocol: Prediction diagnostics compare predicted and physical RMS CV across five matched Qwen resolutions using MAE, maximum error, Pearson correlation, and Spearman correlation.These descriptive values are not significance tests.
  • A.4 Routing and Evaluation Protocol: The fixed routing probe contains 500 seeded Cambrian-training rows, 469,224 image tokens, 118,218 text tokens, and image fraction a = 0.7988.It is a training-corpus probe rather than a held-out test set; physical sweeps use the paired subset of 466 rows containing images.
  • A.5 Result Provenance: Benchmark results use benchmark-specific inputs, diagnostic results use the fixed probe, and physical-shift results use the paired 466-request subset.No probe selects a checkpoint.

B.1 Composition Shift Law

With image and text conditional load profiles fixed, mixed load varies quadratically with image-token fraction, with curvature determined by their profile gap. The law separates composition effects from physical preprocessing changes and explains why mixed balancing can hide conditional imbalance.

  • Composition Shift Law: Fixed conditional profiles make the mixed load profile linear in image-token fraction, so squared load follows a quadratic composition-shift law.The derivation excludes profile changes caused by resolution, tiling, or content.
  • Composition Shift Law: The conditional image-text profile gap sets curvature: larger gaps narrow the low-load region around the balancing fraction.Load grows linearly in |a − a0|, while squared load grows quadratically; if κ = 0, load is constant.
  • Composition Shift Law: ReBA targets conditional imbalance to reduce composition sensitivity, with feasible minima at the unrestricted optimum or nearest tested-interval endpoint.The unrestricted optimum is a⋆; when κ = 0, image and text profiles match at every layer and R(a) is constant.
  • Composition Shift Law: Expected squared deployment load combines a fixed-profile floor, composition variance, and mismatch between mean composition and the best fraction.Controlled AUC uniformly weights a fixed interval and is a standardized range summary rather than deployment expectation.
  • Composition Shift Law: A mixed profile can be uniform while both conditional profiles remain imbalanced, so one mixed balancing term cannot identify both conditional loads.Exact cancellation requires opposite residual directions with composition-dependent magnitude ratios.
  • Composition Shift Law: Physical preprocessing can alter conditional profiles, so actual load may differ from fixed-profile predictions; Qwen mostly preserves structure, whereas InternVL tiling changes it more strongly.Interaction terms can either increase or offset predicted imbalance, allowing physical curves to fall below fixed-profile predictions.

B.2 Separate Modality Terms … C.2 Offline Fixed Probe Checkpoints and Hard and Soft Alignment

The appendices show why ReBA separates modality and image-level balancing: mixed objectives permit cross-modal cancellation, whereas ReBA uniquely penalizes conditional imbalance and can balance both within and across images. Additional checks show lower ReBA load across coefficient settings and saved checkpoints, while hard-soft alignment supports an approximation rather than an identity.

  • B.2 Separate Modality Terms: A mixed objective cannot distinguish oppositely imbalanced image and text profiles whose weighted mixture remains uniform.The image profile shifts by Δ while the text profile shifts oppositely, preserving the mixed profile.
  • B.2 Separate Modality Terms: ReBA’s separate modality terms make conditional imbalance non-canceling, with a unique exact-alignment minimum at qI = qT = u.Each conditional squared CV is nonnegative, and each is bounded by ε divided by its modality weight when total excess is at most ε.
  • B.2 Separate Modality Terms: Separate terms provide modality-specific gradient feedback, sending image signals to image tokens and text signals to text tokens.This separation removes cross-modal cancellation; the appendix motivates the image term’s separate per-image instances.
  • B.3 Image Balance Decomposition: ReBA’s image objective can balance load both within images and across images whose equal-weight mean profile is balanced.The decomposition separates within-image imbalance from variation among image profiles and mean-profile imbalance.
  • C.1 Coefficient Protocol: 0.1 gives the lowest image, text, and overall ReBA load across the tested λaux grid, without establishing optimality outside those values.Each grid value corresponds to a separate training run, with modality weights set to the realized image and text token fractions.
  • C.2 Offline Fixed Probe Checkpoints and Hard and Soft Alignment: ReBA has lower image and text load at every evaluated saved checkpoint on the same fixed 500-sample probe.Overall differences are smaller because image and text errors can cancel under Std-Aux.
  • C.2 Offline Fixed Probe Checkpoints and Hard and Soft Alignment: Most layers show positive hard-soft routing alignment for both methods and modalities, but ReBA’s weaker alignment supports an approximation rather than an identity.Several ReBA layers are close to uniform, making directional cosine less stable because deviation norms are small.

C.3 Expert Compute Proxy · D Workload Shift Results · D.1 Fixed Profile Load Curves

ReBA reduces bottleneck expert work under an idealized compute proxy and achieves the lowest controlled worst load and AUC across the evaluated split backbones. Its physical worst-load advantage holds for Qwen and InternVL, while fixed-profile and physical AUC domains remain distinct.

  • C.3 Expert Compute Proxy: The proxy measures dispatched-token work assigned to the busiest expert across MoE layers under idealized equal-device assumptions.It excludes communication, kernel overhead, shared experts, and non-MoE layers.
  • C.3 Expert Compute Proxy: Perfect balance yields Uproxy = 1, while lower utilization implies more waiting.The proxy compares average expert work with bottleneck work.
  • C.3 Expert Compute Proxy: ReBA gives no proxy speedup at the lowest Qwen setting but gives 1.23 to 1.25 times ideal speedup at the three largest Qwen settings.InternVL gains are smaller and remain above one at every tile count.
  • C.3 Expert Compute Proxy: The proxy supports potential expert compute benefit rather than an end to end speedup.Real execution also depends on communication, expert placement, kernels, shared experts, and non-MoE layers.
  • D Workload Shift Results: Fixed-profile recomposition isolates composition changes, whereas physical forwards may also change the conditional load profiles.Panel (a) isolates composition under fixed profiles; panel (b) reports physical forwards.
  • D.1 Fixed Profile Load Curves: ReBA gives the lowest controlled worst load and AUC on both split backbones.Lower is better for every reported metric, and controlled recomposition uses a ∈[0.1, 0.9].
  • D.1 Fixed Profile Load Curves: ReBA also gives the lowest physical worst load for Qwen and InternVL.Physical sweeps use realized image token fractions.
  • D.1 Fixed Profile Load Curves: Controlled and physical AUC values use different domains and should not be compared as estimates of one deployment distribution.Table S7 defines AUC as the normalized trapezoidal average over each stated domain.

D.2 Qwen Physical Resolution Sweep … E.6 Physical Resolution Sweep and Profile Drift

The appendices validate fixed-profile load-curve structure in some physical sweeps but show that preprocessing can also alter conditional profiles. Native diagnostics attribute sensitivity mainly to conditional load magnitude, while physical resolution changes can flatten or reverse fixed-profile predictions.

  • D.2 Qwen Physical Resolution Sweep: 0.014 MAE and 0.983 Pearson correlation summarize fixed-profile predictions across all 15 Qwen physical-sweep pairs.Std-Aux has MAE 0.006 and Pearson correlation 0.998; Coupled-ImgInst and ReBA preserve the broad predicted trend despite larger residuals.
  • D.2 Qwen Physical Resolution Sweep: ReBA lowers p95 at every tested Qwen resolution, supporting the aggregate load result at request level.The request-level percentiles are computed over per-request RMS CV values for 466 paired requests.
  • D.3 InternVL Tiling Sweep: ReBA has lower physical load than Std-Aux at every tested InternVL tile count, although prediction error grows at six and twelve tiles.Large negative residuals indicate that profile changes offset the composition-only prediction; residuals include image change, text change, and interaction effects.
  • E Native MoE Diagnostics; E.1 Router and Probe Protocol: Native routing probes span an image token fraction of about 0.51 to 0.90 because image tokens change by almost one order of magnitude while text counts stay nearly fixed.The probe uses the same 64 paired samples at low, source, and high resolution.
  • E.2 Source Image and Text Load Gap: Both native models have large conditional image and text loads, while mixed load is smaller and orientation differs between models.Thus, large fixed-profile sensitivity does not require strong negative alignment; conditional magnitude and orientation are separated diagnostically.
  • E.3 Magnitude and Orientation Decomposition: The mean conditional magnitude and orientation terms are 1.780 and +0.084 for 30B, yielding κ = 1.865, while 35B has 2.536 and −0.290, yielding κ = 2.246.Orientation contributes 4.5% for 30B, and conditional magnitude remains large for 35B despite mild alignment.
  • E.4 Finite Count Null: Every observed native load exceeds the count-matched uniform-routing null mean by more than 23 times, and no simulated trial reaches an observed value.The null preserves token count, expert count, top k, and layer count, so this conclusion applies specifically to that matched null.
  • E.5 Fixed Profile Load Curves; E.6 Physical Resolution Sweep and Profile Drift: Both native checkpoints have curved fixed-profile load functions, but physical resolution makes the 30B curve flatter and reverses the predicted high-resolution increase for 35B.Physical resolution changes both composition and conditional profiles, so fixed-profile curves measure composition sensitivity only and do not predict all physical changes.

E.7 Exact Profile Drift and Expert Preference · E.8 Earlier 500 Row Probe with Different Preprocessing

E.7 decomposes physical load changes into fixed-profile load, profile drift, and their interaction, showing that drift—especially image-profile drift—can flatten or reverse composition-only predictions and that native experts exhibit modality preference. E.8 qualitatively replicates orientation differences and nonzero conditional gaps under different preprocessing, while cautioning against pooling results or inferring native ReBA performance.

  • E.7 Exact Profile Drift and Expert Preference: Equation (S18) separates squared physical load into fixed-profile load, drift magnitude, and interaction.Table S15 defines Rpred and Rphys as squared fixed-profile and physical RMS CV, with D and 2Cint capturing drift and interaction.
  • E.7 Exact Profile Drift and Expert Preference: Negative interaction at both high-resolution settings flattens the 30B prediction and reverses the 35B prediction, with image-profile change accounting for most measured drift.The interaction sign determines whether profile change reinforces, flattens, or reverses the composition-only prediction.
  • E.7 Exact Profile Drift and Expert Preference: Profile change can reduce physical load even when composition-only load rises because the interaction term is negative.This explains how conditional profile drift alters the physical load beyond fixed-profile composition effects.
  • E.7 Exact Profile Drift and Expert Preference: 67 of 128 30B experts and 156 of 256 35B experts receive more than twice as much dispatch from one modality as the other.These counts equal 52% and 61% of routed experts, respectively, supporting modality specialization rather than universal strong complementarity.
  • E.8 Earlier 500 Row Probe with Different Preprocessing: The earlier 500-row probe uses text-containing rows and preprocessing different from the fresh 64-sample protocol, serving as a qualitative replication.Table S16 reports conditional RMS CV, median centered residual cosine, fixed-profile curvature, and RMS ℓ2 conditional gap for the earlier native probe.
  • E.8 Earlier 500 Row Probe with Different Preprocessing: The earlier probe reproduces the orientation difference and nonzero conditional gaps, but its values and confidence intervals should not be pooled with the fresh protocol.The protocols differ in preprocessing, so the reported statistics are not directly combinable.
  • E.8 Earlier 500 Row Probe with Different Preprocessing: Native results support the routing diagnosis and fixed-profile boundary but do not test ReBA training on native MoEs.The earlier probe therefore informs diagnosis without establishing native-model ReBA performance.
Loading 2608.00574v1…