PaperlayerView exact revision on arXiv

Source-linked AI summary

Gradient Mirage: Trainable yet Label-Unidentifiable Gradients in Large Language Model Split Learning

Shiyu Miao, Yunlong Mao, Zirui Huang, Liang Yao, Tianshuo Zheng, Yanhui Gu, Fan Liu, Sheng Zhong

arXiv:2608.18767v1cs.CLcs.LG

TL;DR

Gradient matching attacks can recover private labels in LLM split learning because exposed gradients are assumed to faithfully reflect the client’s full-label objective. Gradient Mirage breaks this consistency across objective, direction, and scale while preserving optimization utility, and experiments show substantially stronger protection with comparable fine-tuning performance and a superior privacy-utility trade-off.

  • Problem

    Gradient matching attacks exploit exposed split-interface gradients to reconstruct private label sequences in LLM split learning.

  • Method

    Gradient Mirage decouples learning from disclosure by injecting objective, directional, and scale inconsistencies into released gradients while preserving effective optimization signals.

  • Results

    Gradient Mirage substantially suppresses label reconstruction under comparable fine-tuning performance and achieves a superior privacy-utility trade-off across multiple LLMs and datasets.

  • Takeaways & Limitations

    The results support Gradient Mirage as an effective defense against GMA-SL in autoregressive LLM split learning without discarding backward-signal utility.

  • Takeaways & Limitations

    The study assumes a curious-but-honest server with access to legitimately exchanged activations and gradients, and focuses on autoregressive LLM split learning.

Abstract

from arXiv · show

Gradient matching attacks (GMAs) in LLM split learning (SL) rely on a critical yet underexplored assumption: the gradient exposed at the split interface is a faithful derivative of the client's full-label training objective. This gradient-objective consistency allows a curious server to recover private labels by searching for a sequence whose induced gradient explains the observation. We propose Gradient Mirage, a defense that breaks this consistency without discarding the optimization utility of the backward signal. Our key idea is to induce the adversary to solve a misspecified inverse problem, in which no plausible label sequence in the sequence space can explain the observed gradients. Concretely, Gradient Mirage achieves this by inducing inconsistency across three dimensions: objective, direction, and scale. Selective Autoregressive Supervision derives the exposed gradient from a masked surrogate loss rather than the full-label objective assumed by the attacker; Scale Blinding then applies randomized multiplicative rescaling, obscuring the gradient's natural magnitude; and Directional Privatization further randomizes the gradient direction while preserving its magnitude through the von Mises-Fisher (vMF) mechanism under a directional metric differential privacy guarantee. Crucially, utility is preserved: the Top segment still learns from all target tokens via Dual-Track Backpropagation, the exposed gradient remains informative since each supervised token retains its complete autoregressive context, and Bottom-Gradient Recovery restores the effective gradient for Bottom-segment optimization. Extensive experiments show that Gradient Mirage provides substantially stronger protection than existing defenses under comparable fine-tuning performance, achieving a better privacy-utility trade-off.

1 Introduction

Label-Shielded Split Learning protects local inputs, labels, and losses, yet exposes activation gradients that enable label-sequence recovery through GMA-SL. Gradient Mirage addresses this vulnerability by decoupling model learning from gradient disclosure through objective, directional, and scale inconsistencies while preserving optimization utility.

  • Threat: GMA-SL enables a curious-but-honest server to recover private label sequences by optimizing against activation gradients at the Trunk–Top split interface.These gradients retain explicit batch structure and are closely tied to local supervision, while the cut-layer representation is already available.
  • Setting: Label-Shielded Split Learning keeps private inputs, labels, and supervised loss on the client while outsourcing the compute-intensive Trunk segment.The Bottom and Top segments remain client-side, and the Trunk is server-side.
  • Gradient Mirage: Gradient Mirage identifies gradient–objective consistency as GMA-SL’s key vulnerability and decouples what the model learns from what the exposed gradient reveals.The defense seeks a released signal that remains serviceable for trunk updates while becoming difficult for the adversary to explain.
  • Gradient Mirage: Gradient Mirage injects objective, directional, and scale inconsistencies by combining Selective Autoregressive Supervision, vMF-based Directional Privatization, and randomized Scale Blinding.Dual-Track Backpropagation keeps the full-label loss local at the Top segment while the transmitted gradient derives from a masked surrogate loss.
  • Optimization utility: Gradient Mirage preserves optimization utility through full Top-segment supervision, retained gradient magnitude under vMF privatization, and Bottom-Gradient Recovery for Bottom-segment optimization.The vMF mechanism has a provable lower bound on amplitude signal-to-noise ratio.
  • Evaluation: Extensive experiments across multiple LLMs and datasets show consistently superior privacy–utility trade-offs against adapted GMA defenses and a rigorous sequence-level DP baseline.The paper presents Gradient Mirage as the first defense shown effective against GMA-SL.

2 Preliminary

Label-shielded LLM split learning keeps labels on-device by partitioning the model across client and server segments, but exposes cut-layer gradients that enable gradient-matching label recovery. Under the autoregressive loss, GMA-SL can match samples independently within a batch because only the known normalization constant couples them.

  • Split-learning setup: Label-shielded SL keeps labels on-device while assigning Bottom and Top segments to the client and the Trunk segment to the server.Clients transmit Bottom-segment representations to the server for trunk processing.
  • Gradient-matching attack: GMA-SL observes the gradient with respect to the cut-layer representation and optimizes dummy labels through a differentiable proxy Top segment to match it.The proxy Top parameters remain fixed during inner-loop matching.
  • Autoregressive objective: LLM training uses one-step-shifted next-token targets and a masked autoregressive cross-entropy loss that excludes invalid positions and optionally the final position.Clients compute this loss locally at the Top segment, while the adversary uses the same structure for matching.
  • Batch separability: GMA-SL admits batch-separable matching because optimizing one sample’s dummy labels cannot affect another sample’s loss terms.The sole batch-level coupling is the normalization N = Σ_bΣ_ℓ M_b,ℓ+1, which the adversary knows because the attention mask is transmitted to the server.

3 Threat Model

The threat model considers a curious-but-honest server that passively logs legitimate split-learning information and performs offline inference on private sequences. The adversary has semi-white-box access to the architecture and pretrained checkpoint, but not exact per-step fine-tuned parameters or optimizer state, and may use gradient-discrepancy objectives for matching.

  • Adversary Model: The server faithfully executes split learning while logging legitimately available information for offline reconstruction of private sequences.This models a passive inference adversary rather than an actively protocol-deviating server.
  • Attack Objective: GMA-SL may match exposed gradients using cosine, ℓ1, ℓ2, or weighted combinations of gradient-discrepancy measures.TAG is given as an example of a weighted combination.
  • Adversary Knowledge: Under the semi-white-box model, the adversary knows the model architecture and pretrained checkpoint but lacks exact per-step fine-tuned parameters and optimizer state.This access level follows prior work on LLM split learning.

4 Gradient Mirage

Gradient Mirage protects LLM split learning by making exposed gradients inconsistent with the attacker’s assumed full-label objective in objective, scale, and direction. It preserves optimization utility through selective supervision, magnitude-preserving directional perturbation, and Bottom-gradient scale recovery.

  • Defense overview: Gradient Mirage induces objective, direction, and scale inconsistency so no exposed gradient faithfully matches the attacker’s assumed full-label training objective.Selective Autoregressive Supervision uses a masked loss, Directional Privatization randomizes direction, and Scale Blinding randomizes magnitude.
  • Scale inconsistency: Scale Blinding independently reweights supervised-token gradients with positive random coefficients, obscuring their natural token-wise scale and hindering gradient matching.The method uses Unif[1500, 2000] as a broad effective range and adds negligible computational overhead.
  • Objective inconsistency: Selective Autoregressive Supervision selects structured token subsets using entropy-aware grouping, masking most groups while fully supervising one group per step.The groups are formed from high-, medium-, and low-entropy strata, and the supervision pattern varies across steps while maintaining sequence coverage.
  • Objective inconsistency: SAS preserves a coherent optimization signal while making the exposed loss differ from standard full-label autoregressive supervision.One token group receives full supervision at each step, while remaining groups use entropy-aware partial sampling.
  • Directional inconsistency: The vMF mechanism perturbs only gradient direction while preserving magnitude, avoiding clipping-threshold selection and uniformly bounding amplitude distortion.For n = ˜g −g, every realization satisfies ∥g∥2/∥n∥2 ≥1/2.
  • Utility preservation: Bottom-Gradient Recovery normalizes the gradient propagated to the Bottom segment by the expected Scale Blinding factor before optimization.It compensates for the deliberate amplification introduced by token-level gradient reweighting.

5 Experiments

Experiments across three 7B–8B language models and three datasets show that Gradient Mirage preserves fine-tuning utility while providing stronger protection than gradient pruning, dropout, and GradSeq-LDP. Ablations further identify randomized scale and token-wise variation as effective defenses without materially impairing optimization.

  • Experimental Setup: Experiments evaluate Gradient Mirage on Llama-2-7B, Llama-3-8B, and DeepSeek-LLM-7B using CodeAlpaca, GSM8K, and PIQA.Models and attack tensors use bfloat16, with six decoder layers in Bottom and five in Top.
  • Defense Comparison: Small GradSeq-LDP privacy budgets produce highly unstable training that underperforms Top-Only Training and can become unstable across random seeds.The instability persists even when an individual run initially appears stable.
  • Ablation Studies: Increasing Scale Blinding’s mean scale rapidly decreases reconstruction quality in the low-scale regime while leaving fine-tuning utility largely unaffected.The privacy benefit gradually stabilizes once the scale reaches a sufficiently large range.
  • Ablation Studies: Token-wise scale variation consistently improves Scale Blinding over small-variation settings, but excessive variation can disturb optimization without proportional privacy gains.The default r = 1/7 balances reconstruction resistance and optimization stability.
  • Ablation Studies: Updating Trunk parameters does not compromise training stability and slightly improves training performance despite perturbations to Trunk gradients.This result supports using perturbed Trunk gradients for model updates.

6 Conclusion

The paper identifies gradient–objective consistency as the root vulnerability enabling GMA-SL in LLM-SL and introduces Gradient Mirage to turn gradient matching into a misspecified inverse problem. The defense injects objective, directional, and scale inconsistencies while preserving full-label learning and recovering the effective bottom gradient.

  • Gradient–objective consistency enables GMA-SL in LLM-SL, making it the root vulnerability targeted by Gradient Mirage.
  • Gradient Mirage injects objective, directional, and scale inconsistencies into the exposed gradient, turning gradient matching into a misspecified inverse problem.
  • The defense decouples what the model learns from what the gradient reveals by preserving full-label learning and recovering the effective bottom gradient.
  • Experiments evaluate Gradient Mirage across multiple LLMs and datasets.

SUPPLEMENTARY MATERIAL … A.2 Rejection Sampling for t

The supplementary material expands the vMF mechanism, Selective Autoregressive Supervision, experiments, visualizations, and future work. Its vMF details explain tangent-normal decomposition and rejection sampling for the axial variable.

  • SUPPLEMENTARY MATERIAL: The supplementary material covers vMF details, Selective Autoregressive Supervision, additional experiments, Gradient Mirage visualizations, and future work.
  • A Details of the vMF Mechanism: The vMF mechanism uses a distribution defined on the unit hypersphere S^n−1.
  • A Details of the vMF Mechanism: The vMF distribution is parameterized by mean direction µ ∈ S^n−1 and concentration parameter κ ≥ 0.
  • A.1 Tangent-Normal Decomposition in vMF Sampling: Tangent-normal decomposition represents sampled direction ˜µ using axial projection t = ˜µ^⊤µ = cos θ and an orthogonal component.
  • A.1 Tangent-Normal Decomposition in vMF Sampling: The orthogonal component has magnitude 1−t^2 = sin θ and uses ξ sampled uniformly from the subsphere orthogonal to µ.
  • A.1 Tangent-Normal Decomposition in vMF Sampling: Thus, vMF sampling separates axial-variable sampling from uniform tangential-direction sampling on the orthogonal subsphere.
  • A.2 Rejection Sampling for t: Rejection sampling generates t ∈ [−1, 1] using a Beta proposal after the tangent-normal decomposition.
  • A.2 Rejection Sampling for t: Each proposed t is accepted using a uniform random variable u ∼ Unif(0, 1); rejected samples trigger repetition before combining t with the tangential direction.

A.3 Proofs related to the vMF Mechanism · B Details of Selective Autoregressive Supervision

The vMF mechanism is proven to satisfy both Euclidean and angular metric differential privacy, while its perturbation norm is tightly bounded. The accompanying illustration shows that masking the last k target tokens induces a zero-gradient suffix in selective autoregressive supervision.

  • A.3 Proofs related to the vMF Mechanism: The vMF mechanism satisfies εd2-metric differential privacy.This is stated as Theorem 2.
  • A.3 Proofs related to the vMF Mechanism: The vMF mechanism also satisfies εd∠-metric differential privacy.The result is stated as Theorem 3, with the probability bound expressed using angular distance.
  • A.3 Proofs related to the vMF Mechanism: The angular-privacy proof derives the result from Theorem 2 by showing ∥u − v∥2 ≤ d∠(u, v) for unit vectors.The proof sets θ = d∠(u, v) ∈ [0, π] and substitutes the distance inequality into the Euclidean privacy bound.
  • A.3 Proofs related to the vMF Mechanism: For g = ∥g∥2u and ˜g = ∥g∥2˜u, the perturbation norm equals ∥n∥2 = ∥g∥2∥˜u − u∥2.The perturbation is represented through the difference between the original and privatized unit directions.
  • A.3 Proofs related to the vMF Mechanism: The perturbation norm is at most 2∥g∥2, and this bound is tight when ˜u = −u.Equality occurs exactly when the two unit directions are opposites.
  • B Details of Selective Autoregressive Supervision: Masking the last k target tokens in selective autoregressive supervision induces a zero-gradient suffix.The figure depicts gradient propagation arrows for this masked suffix.

B.1 Last-k Tokens in Selective Autoregressive Supervision · C More Experimental Details · C.1 Definitions of Gradient Utility Metrics

SAS must avoid masking all last-k target tokens, which otherwise creates a detectable zero-gradient suffix; gradient utility is evaluated through magnitude- and support-preservation metrics.

  • B.1 Last-k Tokens in Selective Autoregressive Supervision: Masking the last k consecutive target tokens can make corresponding suffix representations receive identically zero gradients because their downstream supervised losses are all masked.This follows from causal autoregressive dependencies, where late representations affect only aligned or future prediction positions.
  • C More Experimental Details: Figure 14 reports learning-based inversion-decoder training loss, evaluation loss, and evaluation RougeL-F across five LLMs and three random seeds.These training dynamics are included among the paper’s additional experimental details.
  • B.1 Last-k Tokens in Selective Autoregressive Supervision: The resulting contiguous zero-gradient suffix is a detectable, token-content-independent artifact that can reveal selective masking from interface-gradient norms.The artifact weakens SAS concealment by exposing a structural signature to the server.
  • B.1 Last-k Tokens in Selective Autoregressive Supervision: SAS therefore requires at least one supervised token among the last k positions; experiments enforce all last five positions as valid supervised tokens.The implementation sets k = 5 to prevent a fully zero-gradient suffix.
  • C.1 Definitions of Gradient Utility Metrics: Gradient utility is quantified with ASNR, ASNR@10%, Recall@10% (R@10%), and Jaccard@10% (J@10%) using original, privatized, and noise gradients.The injected noise is defined as n = ˜g − g.
  • C.1 Definitions of Gradient Utility Metrics: ASNR and ASNR@10% measure magnitude preservation, whereas R@10% and J@10% measure preservation of the original gradient’s salient support.Support metrics compare top-10% coordinate index sets ranked by absolute gradient value.

C.2 Training Details of the Learning-based Inversion Decoder … C.5 Extended Defense Experiments

The appendix specifies the SIP inversion decoder’s training setup, shows instability for strong gradient pruning/dropout protection, and confirms benefits from Trunk updates. It also extends defense evaluation across additional metrics, models, training durations, and BiSR(b) cosine matching.

  • C.2 Training Details of the Learning-based Inversion Decoder: SIP reconstructs private input tokens from Bottom-segment smashed data using an inversion decoder.The decoder is trained for the semi-white-box Forward Inversion Paradigm (SIP-only) setting.
  • C.2 Training Details of the Learning-based Inversion Decoder: The decoder trains on CNN-DailyMail auxiliary summarization data, using a 6-layer Bottom segment and a single-layer GRU with hidden size 4096.The GRU models sequential dependencies in intermediate activations, followed by a linear projection to vocabulary logits.
  • C.2 Training Details of the Learning-based Inversion Decoder: Training freezes the Bottom segment as an encoder and optimizes the inverter with token-level cross-entropy for 3 epochs using AdamW.The stated optimizer settings are learning rate 1e-3, weight decay 1e-5, and batch size 32; the trained inverter reconstructs observed smashed data.
  • C.3 Training Curves of GP and GD: Strong privacy protection from Gradient Pruning or Gradient Dropout makes training extremely unstable and weakens the privacy-utility trade-off.This finding comes from the training curves for GP and GD.
  • C.4 Ablation Details of Trunk Training: Updating the Trunk segment consistently yields better performance across three datasets and random seeds on Llama3-8B.The comparison evaluates training dynamics with and without Trunk segment updates.
  • C.5 Extended Defense Experiments: Extended experiments report ROUGE-1-F, ROUGE-2-F, and Token Recovery Rate alongside ROUGE-L-F and METEOR over more training steps.The results cover Llama-3-8B, Llama-2-7B, and DeepSeek-LLM-7B.
  • C.5 Extended Defense Experiments: Additional evaluation tests Gradient Mirage against the BiSR(b) paradigm when cosine distance is used as the matching objective.This comparison is reported in Table 11.

C.6 Robust Training under Extreme Scale Blinding · D Visualization of Gradient Mirage Performance

Under extreme Scale Blinding, Gradient Mirage remains trainable by freezing the Trunk while optimizing Top normally and updating Bottom with recovered gradients. Visualizations across models show substantially more distorted, incoherent reconstructions that reveal less ground-truth information than existing defenses.

  • C.6 Robust Training under Extreme Scale Blinding: As the mean scale m grows excessively large, amplified gradients can destabilize optimization, especially in the directly affected Trunk segment.The default mean scale is m = 1750.
  • C.6 Robust Training under Extreme Scale Blinding: Trunk-Frozen Training discards Trunk gradients, optimizes Top normally, and updates Bottom with recovered gradients that remove Scale Blinding amplification.The recovered Bottom gradients remain relatively stable and informative for optimization.
  • C.6 Robust Training under Extreme Scale Blinding: Stable convergence persisted across m ∈{4×103, 104, 2.5×104, 5×104, 105}, consistently outperforming Top-Only Training.The evaluation used significantly larger mean scales than the default setting.
  • D Visualization of Gradient Mirage Performance: Under instruction fine-tuning on CodeAlpaca, PIQA, and GSM8K, GMA-SL reconstructs the supervised answer portion even when query tokens are excluded from user-side supervision.The qualitative results concern GMA-SL behavior rather than Gradient Mirage’s defense output.
  • D Visualization of Gradient Mirage Performance: Gradient Mirage yields substantially more distorted and incoherent reconstructions than existing privacy-preserving defenses across Llama-3-8B, Llama-2-7B, and DeepSeek-LLM-7B.The qualitative comparison covers representative defense performance across three large language models.
  • D Visualization of Gradient Mirage Performance: These reconstructions reveal considerably less information about the ground-truth input, visually demonstrating Gradient Mirage’s effectiveness against GMA-SL.The conclusion is drawn from representative qualitative examples across three LLMs.

E Future Work

The section shows that GMA-SL remains a substantial threat under instruction fine-tuning despite supervision being restricted to answer tokens. It identifies multimodal privacy, attack mechanisms, and reinforcement-learning settings as future research directions.

  • Instruction Fine-Tuning: GMA-SL remains effective under instruction fine-tuning, where user-side supervision is applied only to answer tokens rather than the entire sequence.The attack evaluation otherwise constructs labels by left-shifting the full sequence, supervising all tokens.
  • Instruction Fine-Tuning: Despite the full-supervision attack mismatch, answer reconstruction quality remains remarkably high, while query reconstruction quality is relatively poor.Representative examples are reported for CodeAlpaca, GSM8K, and PIQA.
  • Future Directions: Future work will extend Gradient Mirage to multimodal large language models and investigate GMA-SL mechanisms, its instruction-fine-tuning effectiveness, and reinforcement-learning settings without explicit token-level labels.The paper specifically highlights why GMA-SL remains effective when supervision is restricted to answer tokens.
Loading 2608.18767v1…