Source-linked AI summary
When Routing Reveals Membership: Privacy Leakage from MoE Router Telemetry
Yixin Tan, Jiayang Liu, Lu Sun, Yuke Hu, Zheng Li, Rui Wen
TL;DR
Exposed router telemetry may reveal whether an example was used to fine-tune a deployed MoE model, a question not resolved by output-based attacks or typically inaccessible internal-state signals. The paper combines output-side signals with routing features and finds added membership inference signal across architectures and domains. The results identify telemetry as an additional privacy surface in fine-tuned MoE deployment.
Problem
Whether exposed router telemetry can reveal fine-tuning membership remains largely unexplored, while output-based attacks may capture only limited membership information at low false-positive rates.
Source: §1 · p. 1Method
The attack combines output-side membership signals with aggregated router telemetry and trains a classifier using independently fine-tuned shadow models on same-domain auxiliary data.
Source: §1 · p. 2Results
Router telemetry improves TPR @1% FPR over the output-signal ensemble in all nine model–domain settings, with absolute gains from 0.0267 to 0.0937.
Source: §5.2 · p. 6Takeaways & Limitations
Router telemetry constitutes an additional privacy surface because it exposes membership signal beyond conventional output-side attacks.
Source: §1 · p. 3, §6 · p. 9Experiments use open-weight models and constructed arXiv, GitHub, and Wikipedia membership instances, not human subjects or private user data from deployed systems.
Source: ETHICS STATEMENT · p. 10
Abstract
from arXiv · showhide
Mixture-of-Experts (MoE) language models produce routing information during inference that may be logged or exposed for monitoring, debugging, load analysis, and safety auditing. Unlike ordinary model outputs, this telemetry reveals a view of the model's internal computation, raising a privacy question: can it reveal whether an example was used to fine-tune the deployed model? We introduce a router-augmented membership inference attack that combines conventional output-side signals with aggregated routing features and applies a membership classifier learned from independently fine-tuned shadow models to the target model. Across three MoE architectures and three data domains, router telemetry consistently improves membership inference over a strong output-signal ensemble, increasing TPR at 1\% FPR by 2.7--9.4 percentage points across all nine settings. The leakage persists across full fine-tuning, frozen-router training, LoRA, and instruction tuning, and remains observable with only discrete expert selections, restricted telemetry, or a single shadow model. Mechanistic analysis further shows that the leakage does not require router-specific memorization: fine-tuning introduces membership information into hidden representations, while the router exposes a projection of this signal even when its parameters are frozen. Perturbing the telemetry reduces this additional leakage only as its fidelity degrades. Our results show that router telemetry can turn an operational signal into an additional privacy surface for fine-tuned MoE models.
1 INTRODUCTION
The introduction asks whether exposed router telemetry can reveal fine-tuning membership beyond model outputs, addressing a gap between output-only attacks and internal signals that are usually inaccessible. It proposes combining routing features with output signals and reports that this adds leakage across architectures and domains.
1 INTRODUCTION
- The privacy question is whether telemetry exposed during MoE inference can reveal that a candidate input was used to fine-tune the deployed model.The telemetry may be retained or surfaced for monitoring, debugging, load analysis, or safety auditing.Source: §1 · p. 1
- Output-based attacks may miss membership information at stringent low-false-positive operating points, while intermediate-state attacks generally require access unavailable in standard deployment.Router telemetry occupies an intermediate regime: it derives from internal representations but may be exposed operationally during inference.Source: §1 · pp. 1–2
- The contribution frames router telemetry as an additional privacy surface beyond conventional model outputs.The attack combines output-side signals with aggregated routing features and learns from same-domain auxiliary data.Source: §1 · p. 2
- The proposed attack combines output-side membership signals with aggregated router telemetry and trains its classifier on independently fine-tuned, same-domain shadow models.The classifier is then applied to the target model.Source: §1 · p. 2
- Router telemetry consistently adds membership signal beyond output-side attacks across three MoE architectures and three data domains.The introduction also reports persistence across fine-tuning regimes and restricted attacker access, with leakage tied to hidden-representation information rather than router-specific memorization.Source: §1 · pp. 2–3
2 RELATED WORK
Prior work studies membership inference from model outputs and internal representations, as well as information leakage from MoE routing. This paper addresses whether exposed router telemetry reveals membership in fine-tuning data.
2 RELATED WORK
- Output-based membership attacks use likelihood, confidence, reference-checkpoint comparisons, or shadow-model distributions, but capture only signals reflected in predictions.Examples include Min-K% (Shi et al., 2024), LiRA (Carlini et al., 2022), WBC (Chen et al., 2026), and LTsig (Ilic et al., 2026a).Source: §2 · p. 3
- Representation-based attacks use hidden states, attention patterns, or gradient-induced changes, but typically require access to internal model information unavailable in deployment.Examples include Makhija et al. (2026), Zaree et al. (2026), and Ranjan et al. (2026).Source: §2 · p. 3
- Prior routing studies infer input properties or exploit hardware side channels, but do not examine membership leakage from exposed telemetry itself.GateBleed studies membership inference through hardware timing and requires hardware-level side-channel access.Source: §2 · p. 3
3 PROBLEM STATEMENT
The problem statement defines routing information and membership inference for fine-tuned MoE models, then specifies the deployment threat model and evaluation measures.
3 PROBLEM STATEMENT
- A Mixture-of-Experts model replaces some or all dense feed-forward blocks with expert networks and a learned router that selects top-k experts for each token.Router telemetry may include continuous scores or probabilities, selected-expert identities, or aggregated expert-usage statistics.Source: §3 · pp. 3–4
- Membership inference asks whether a candidate example was included in the private fine-tuning dataset, and the attack assigns a score thresholded into a membership prediction.The paper reports AUROC and uses TPR at 1% FPR as its primary low-FPR metric.Source: §3 · p. 4
3.1 THREAT MODEL
- The attacker knows the pretrained checkpoint and fine-tuning-data domain, observes model outputs and router telemetry, and can train shadow models on same-domain auxiliary data.The attacker lacks the private dataset and access to the target model’s parameters, hidden states, and gradients.Source: §3.1 · p. 4
4 ROUTER-AUGMENTED MEMBERSHIP INFERENCE
The attack augments output-derived membership signals with a fixed-dimensional representation of routing behavior. It learns a membership classifier from shadow models fine-tuned on same-domain auxiliary data and applies that classifier to the target.
4 ROUTER-AUGMENTED MEMBERSHIP INFERENCE
- The attack combines output-derived membership signals with a fixed-dimensional routing representation and applies a shadow-trained membership classifier to the target model.Shadow models are independently fine-tuned on auxiliary data from the same domain.Source: §4 · p. 4
- The output-side representation ensembles signals from the target fine-tuned model with reference-based signals comparing it to the public pretrained checkpoint.This representation characterizes the membership information available from conventional model outputs.Source: §4 · p. 5
- Router features summarize each input by averaging the routing probability assigned to each expert and concatenating the statistics across layers and experts.The result is a fixed-dimensional router representation.Source: §4 · p. 5
- The attacker constructs shadow-model training and held-out sets from same-domain auxiliary data, extracts output and routing representations, and pools them to train the membership classifier.The learned classifier is applied directly to the independently fine-tuned target model.Source: §4 · p. 5
5 EXPERIMENTS
The experiments test whether router telemetry adds membership-inference signal beyond model outputs across MoE models, domains, adaptation regimes, and restricted-access settings. They also examine where the signal appears and whether perturbing telemetry reduces it.
5.1 EXPERIMENTAL SETUP
- The evaluation uses three MoE models across arXiv, GitHub, and Wikipedia, with leave-one-out testing across independently fine-tuned target and shadow models.Output-only and router-augmented classifiers are trained on shadow-model data and evaluated on held-out targets.Source: §5.1 · pp. 5–6
5.3 LEAKAGE PERSISTS ACROSS FINE-TUNING REGIMES
- Router telemetry improves TPR @1% FPR under full fine-tuning, frozen-router training, LoRA, and instruction tuning, with gains from 0.0305 under LoRA to 0.0904 with a frozen router.Freezing the router does not reduce the additional leakage compared with full fine-tuning.Source: §5.3 · p. 6, Table 2 · p. 7
5.2 ROUTER TELEMETRY EXPOSURE AMPLIFIES MEMBERSHIP LEAKAGE
- TPR @1% FPR improves by 0.0267 to 0.0937 across all nine model–domain settings when router telemetry augments Aout.The gain is more pronounced at this low-FPR operating point than the AUROC gains.Source: §5.2 · p. 6, Table 1 · p. 7
5.4 LEAKAGE UNDER RESTRICTED TELEMETRY AND SHADOW-MODEL ACCESS
- Discrete expert selections improve TPR @1% FPR in all nine model–domain settings, with gains from 0.0283 to 0.1043, comparable to continuous telemetry.The result indicates precise routing probabilities are not needed to expose a comparable membership signal.Source: §5.4 · p. 7, Table 3 · p. 8
- Top-1 telemetry adds 0.0175 TPR @1% FPR, rising to 0.0564 at top-K = 8 and 0.0641 at top-K = 32; one shadow model still adds 0.0493.The top-K gains increase monotonically with diminishing improvements as more experts are exposed.Source: §5.4 · p. 8, Figure 2 · p. 8
5.5 ROUTER TELEMETRY EXPOSES MEMBERSHIP SIGNALS IN HIDDEN REPRESENTATIONS
- After two fine-tuning epochs, router and matched-rank random projections reach AUROC 0.661 and 0.594, while the router projection reaches 0.661 with either full or frozen-router fine-tuning.The findings suggest fine-tuning adds membership information to hidden representations and router telemetry exposes a projection of it, without requiring router-specific memorization.Source: §5.5 · p. 8, Figure 3 · p. 8
5.6 MITIGATING ROUTER-TELEMETRY LEAKAGE TRADES OFF UTILITY
- Stronger Gaussian noise, feature dropout, and quantization reduce additional membership leakage while degrading telemetry utility; within the evaluated settings, no high-fidelity configuration suppresses the leakage.The strongest suppression under dropout and quantization occurs when telemetry correlation is close to zero.Source: §5.6 · p. 9, Figure 4 · p. 9
6 CONCLUSION
Router telemetry adds membership signal beyond model outputs across the studied MoE settings, without requiring router-specific memorization. Perturbing telemetry reduces leakage as its fidelity degrades, making routing information a privacy consideration in deployment.
6 CONCLUSION
- Router telemetry provides membership signal beyond conventional output-side attacks across multiple architectures, domains, and fine-tuning regimes.Discrete selections, restricted top-K telemetry, and limited shadow-model access also remain sufficient to improve inference.Source: §6 · p. 9
- Fine-tuning introduces membership-related structure into hidden representations, which the router exposes even when its parameters are frozen.The conclusion says the effect is not explained by router-specific memorization.Source: §6 · p. 9
- Telemetry perturbation reduces additional leakage, but stronger perturbations also degrade telemetry fidelity.Source: §6 · p. 9
AI USE STATEMENT
Generative AI tools assisted with manuscript polishing, and the authors reviewed and revised the resulting text. The tools were not used to generate research ideas, results, or scientific claims.
AI USE STATEMENT
- Generative AI tools assisted with grammar, clarity, and readability edits to the manuscript.Source: AI USE STATEMENT · p. 10
- The authors reviewed and revised all AI-assisted text.Source: AI USE STATEMENT · p. 10
- AI tools were not used to generate research ideas, experimental results, or scientific claims; the authors take responsibility for the final content.Source: AI USE STATEMENT · p. 10
ETHICS STATEMENT
The work examines the dual-use risk of inferring fine-tuning membership from router telemetry. Its experiments use controlled open-weight-model settings and constructed datasets, not private user data from deployed systems.
ETHICS STATEMENT
- Membership inference techniques have dual-use potential because they can reveal whether examples appeared in fine-tuning data.Source: ETHICS STATEMENT · p. 10
- Experiments use open-weight models and constructed arXiv, GitHub, and Wikipedia membership instances, without human subjects or private user data from deployed systems.Source: ETHICS STATEMENT · p. 10
- The work evaluates telemetry perturbations to measure the trade-off between reducing membership leakage and preserving operational utility.Source: ETHICS STATEMENT · p. 10
A.1 DATASET CONSTRUCTION AND VALIDATION
The appendix describes dataset validation, model fine-tuning, output-side attack features, router telemetry representations, and a leave-one-out shadow-model evaluation protocol.
A.1 DATASET CONSTRUCTION AND VALIDATION
- Four membership instances are constructed per domain without sharing documents across instances; 48 text-only checks yield AUROC values from 0.4817 to 0.5329.The checks indicate that membership labels are not readily predictable from surface properties alone.Source: §A.1 · p. 14
A.2 MODELS AND FINE-TUNING
- Three open-weight MoE models are evaluated across three domains, with four independently fine-tuned models per setting for leave-one-out target and shadow roles.The main experiments use full-parameter fine-tuning, including router parameters.Source: §A.2 · p. 14
A.3 MIA BASELINE IMPLEMENTATION
- The output-side baseline combines likelihood and confidence features with reference-based comparisons, ReCaLL, and an LTsig-style signature for a histogram gradient boosting classifier.The LTsig-style feature uses a fixed-dimensional summary rather than the original sequence-based classifier.Source: §A.3 · pp. 15–16
A.4 ROUTER-TELEMETRY REPRESENTATION
- Continuous telemetry averages each expert’s softmax routing probability across tokens and flattens per-layer loads; the discrete alternative records expert-selection frequencies.Source: §A.4 · pp. 16–17
A.5 ATTACK TRAINING AND EVALUATION
- The router-augmented attack uses the same leave-one-out shadow-model protocol as the output-only attack, isolating the additional signal from telemetry.Both attacks use the same shadow models, membership labels, classifier family, and held-out target.Source: §A.5 · p. 17
B RESULTS ACROSS MIA BASELINES
Router telemetry adds membership signal across output-side attack baselines, while the supplied sections also describe fine-tuning regimes and restricted telemetry representations used to test the attack.
B RESULTS ACROSS MIA BASELINES
- The baseline comparison tests continuous expert-load telemetry added to individual output-side attacks across the full 3 × 3 model–domain grid.Strong output-only baselines such as LTsig and Aout generally show smaller gains, leaving less headroom for additional improvement.Source: B RESULTS ACROSS MIA BASELINES · p. 17
- Router telemetry improves AUROC in 107 of 108 baseline–model–domain combinations and TPR @1% FPR in 104 of 108.Gains occur across likelihood-, confidence-, and reference-based baselines; the few negligible or negative gains concentrate in logit margin and ReCaLL.Source: B RESULTS ACROSS MIA BASELINES · p. 17, Table 5 · p. 18
C FINE-TUNING REGIMES
- The regime study uses OLMoE-1B-7B on arXiv after one epoch, comparing full fine-tuning, frozen-router training, LoRA, and instruction tuning.Full fine-tuning updates router gates; frozen-router training and LoRA keep them fixed, while instruction tuning uses an instruction–response format.Source: C FINE-TUNING REGIMES · p. 19
D.1 TOP-K ROUTER TELEMETRY
- Top-K telemetry retains only the dimensions for the most heavily loaded experts, while the count representation is constructed from discrete expert selections.The restricted representation follows the same attack training and evaluation protocol as the full expert-load representation.Source: §D.1 · p. 19
D.2 NUMBER OF SHADOW MODELS
The section tests how shadow-model availability, representation projections, and telemetry perturbations relate to router-augmented membership inference.
D.2 NUMBER OF SHADOW MODELS
- 0.0493 TPR @1% FPR improvement over Aout is achieved with one shadow model, rising to 0.0696 with two and 0.0841 with three.The classifier is trained on selected independently fine-tuned models and evaluated directly on a held-out target.Source: Figure 5 · p. 20, §D.2 · p. 20
E MECHANISM ANALYSIS
- The mechanism analysis records hidden states and router weights before fine-tuning and after one and two epochs on member and non-member examples.It uses OLMoE-1B-7B on arXiv and the same four disjoint membership instances as the main experiments.Source: E MECHANISM ANALYSIS · p. 20
E.1 ROUTER-SUBSPACE PROJECTION
- The projection analysis measures membership information in hidden states projected onto the router-weight row space, against a matched-rank random-subspace control.Projected features are aggregated over the sequence, and AUROC is averaged over four models.Source: §E.1 · p. 20
E.2 FROZEN-ROUTER CONTROL
- The frozen-router control repeats the projection analysis while holding router gates fixed and updating the remaining model parameters.Comparing this with full fine-tuning separates information exposed through the router subspace from information arising through direct router-parameter updates.Source: §E.2 · pp. 20–21
F TELEMETRY PERTURBATION DEFENSES
- Telemetry perturbations are applied to the continuous expert-load representation before exposure, while the model and output-side signals remain unchanged.The analysis also measures top-1 expert agreement as a telemetry-utility measure.Source: F TELEMETRY PERTURBATION DEFENSES · p. 21
F.1 PERTURBATION MECHANISMS
- The perturbation mechanisms are Gaussian noise, independently sampled feature dropout, and quantization, evaluated by replacing ϕR in the same attack pipeline.Smaller quantization resolution corresponds to coarser telemetry.Source: §F.1 · p. 21
F.2 TOP-1 EXPERT AGREEMENT
Stronger telemetry perturbations reduce the additional membership signal alongside measures of retained expert-selection information.
F.2 TOP-1 EXPERT AGREEMENT
- At p = 0.9, additional TPR @1% FPR is 0.0190 and top-1 agreement is 0.101; at p = 0.99, they are 0.0040 and 0.050.Feature dropout produces a sharper reduction in telemetry utility.Source: §F.2 · p. 21
- At σ = 4, additional TPR @1% FPR falls from 0.0841 to 0.0159, while top-1 expert agreement falls from 1.000 to 0.407.Figure 6 plots additional TPR @1% FPR and top-1 expert agreement against telemetry perturbation.Source: §F.2 · p. 21, Figure 6 · p. 22
- With B = 8, additional TPR @1% FPR is 0.0121 and top-1 agreement is 0.316; with B = 4, they are 0.0079 and 0.076.Smaller B corresponds to coarser telemetry in the quantization mechanism.Source: §F.2 · p. 22
- Across perturbations, settings that suppress most additional membership signal also substantially degrade the expert-selection information retained by telemetry.Higher top-1 agreement indicates better preservation of expert-selection information from the original router loads.Source: §F.2 · pp. 21–22