PaperlayerView exact revision on arXiv

Source-linked AI summary

LightSplit: Practical Privacy-Preserving Split Learning via Orthogonal Projections (Full Version)

Mert Cihangiroglu, Alessandro Pegoraro, Phillip Rieger, Antonino Nocera, Ahmad-Reza Sadeghi

arXiv:2605.13265v2cs.LGcs.CR

TL;DR

Split learning's cut-layer activations create communication overhead and expose representations to reconstruction attacks, while existing approaches typically address efficiency or privacy in isolation. LightSplit projects activations into a fixed lower-dimensional orthogonal subspace; across benchmarks, it reduces reconstruction capability by up to 7–8× and communication overhead by up to 32× while retaining 86–100% of accuracy.

  • Problem

    Existing split-learning methods typically address communication efficiency or privacy in isolation, despite the cut-layer interface's high-dimensional activations and reconstruction vulnerability.

    Source: §I · pp. 1–2

  • Method

    LightSplit projects cut-layer activations into a fixed, randomly chosen orthonormal subspace, discarding components in the orthogonal complement to limit information available for reconstruction.

    Source: §I · p. 2

  • Results

    Up to 7–8×, LightSplit reduces reconstruction capability across multiple architectures and datasets, while reducing communication overhead up to 32× and retaining 86–100% of accuracy.

    Source: §I · p. 2

  • Takeaways & Limitations

    The projection reduces per-sample payload without adding trainable client parameters.

    Source: E. Positioning Our Approach · p. 14

    LightSplit does not protect against white-box inference attacks that exploit shared model updates, a separate threat outside its smashed-data focus.

    Source: B. Limitations · p. 12

Abstract

from arXiv · show

Split learning (SL) enables collaborative training by partitioning a neural network across clients and a central server. However, the cut-layer interface introduces a key challenge: high-dimensional activations incur substantial communication overhead while exposing representations vulnerable to reconstruction attacks. Existing approaches typically address efficiency or privacy in isolation. We propose LightSplit, which limits information exposure and reduces communication overhead via a lightweight fixed orthogonal random projection at the cut layer. Based on Shannon's information theory, this projection acts as an information bottleneck that restricts instance-specific information and suppresses exploitable per-sample signals. With low-dimensional projections sent instead of raw activations, the server operates on lifted representations without requiring changes to existing SL architectures. Adding no trainable client components, LightSplit remains lightweight and suitable for edge devices while preserving end-to-end differentiability via exact gradient propagation. As the projection is non-invertible, part of the original representation is irreversibly discarded at the client, so LightSplit reduces the information available for reconstruction. We evaluate LightSplit on seven benchmarks spanning image classification, tabular data, and language-model fine-tuning, under IID and non-IID client data. At up to 32x compression it retains at least 94% of baseline accuracy on CIFAR-10, FashionMNIST, GTSRB, and UCI HAR, 86% on CIFAR-100, and 91% on average under non-IID client data; at 8x it retains 92% of ImageNet-1k top-1 and raises GPT-2 perplexity by at most 1.15x. Against state-of-the-art reconstruction attacks LightSplit resists inversion; the strongest, co-trained attack loses up to 87.3% of its reconstruction SSIM, and subject re-identification on UCI HAR drops by more than half.

I. INTRODUCTION … B. Random Projections and Johnson-Lindenstrauss Lemma

LightSplit addresses split learning’s communication and reconstruction risks with fixed low-dimensional orthogonal projections, while within-class compaction helps preserve task utility. The background explains that random projections can preserve geometric structure but do not by themselves guarantee privacy.

A. Split Learning

  • Split learning keeps sensitive inputs on clients but exchanges smashed activations with the server, creating communication costs and exposing representations to reconstruction and other attacks.Source: §I · p. 1

I. INTRODUCTION

  • LIGHTSPLIT projects cut-layer activations into a fixed random orthonormal subspace, reducing transmitted data and irreversibly discarding information that could aid reconstruction.Source: §I · p. 2
  • Within-Class Compaction encourages client representations to align with the surviving subspace, preserving task-relevant information and competitive accuracy under non-IID data.Source: §I · p. 2
  • The fixed client projection requires one matrix multiplication and no additional learnable parameters, while server lift-back modes trade flexibility against computational efficiency.Source: §I · p. 2
  • LIGHTSPLIT’s projections emphasize class-level structure and can improve separability in some scenarios, with potential relevance to detecting model-poisoning attacks.Source: §I · p. 2
  • At 16×, SDAR reconstruction SSIM falls from 0.83 to 0.11; across evaluations LIGHTSPLIT reduces communication up to 32× while retaining 86–100% accuracy.Source: §I · p. 2

B. Random Projections and Johnson-Lindenstrauss Lemma

  • The JL lemma says random orthonormal projections preserve pairwise distances among finite point sets with high probability, retaining activation geometry without task-adaptive optimization.Source: B. Random Projections and… · p. 3
  • JL-style dimensionality reduction alone does not ensure privacy under repeated or auxiliary-informed observations, motivating LightSplit’s within-class regularization and empirical evaluation against adaptive attackers.Source: B. Random Projections and… · p. 3

III. PROBLEM SETTING … B. Client-Side Projection

LightSplit replaces the transmitted cut-layer activation with a lower-dimensional representation to reduce communication and limit information exposure. Its fixed orthonormal projection is designed to preserve task-relevant geometry while reducing the information available to a curious server.

A. System Model

  • In the U-shaped SL setting, clients hold heads and tails around a shared server backbone, and each step transmits a d-dimensional cut activation and returns its gradient.Aggregate bandwidth therefore scales linearly with activation dimension and client count; the goal is a k-dimensional summary with k ≪ d.Source: A. System Model · p. 3

B. Threat Model

  • The honest-but-curious server follows the protocol but may infer private samples from compressed activations, returned gradients, and its evolving backbone parameters.The server observes ˜z ∈ R^k rather than the original z ∈ R^d.Source: B. Threat Model · p. 3

C. Requirements and Challenges

  • Practical SL compression must deter inference, preserve baseline utility and stable convergence, and reduce communication, while avoiding uncoordinated client compression, privacy-conflicting fidelity, and added client training costs.A shared backbone requires globally consistent representations; compression should retain inter-class geometry while discarding reconstruction-enabling per-sample variation.Source: C. Requirements and Challenges · pp. 3–4

IV. LIGHTSPLIT

  • LightSplit projects smashed data into a lower-dimensional subspace to reduce communication and limit information exposure.The projection is applied at the head-backbone cut, while the server operates on lifted representations.Source: §IV · p. 4, A. Design Rationale · p. 4

A. Design Rationale

  • Random projection approximately preserves pairwise distances and class separability, while LIGHTSPLIT-F uses a fixed lift-back and LIGHTSPLIT-L uses a trainable reconstruction network.The fixed variant is parameter-free and lightweight; the learned variant is more flexible and can adapt to projected representations.Source: A. Design Rationale · p. 4, B. Client-Side Projection · p. 5
  • Within-Class Compaction reduces intra-class variation to discourage reconstruction-exploitable feature scattering while preserving class-level structure; the head-backbone-cut design also applies to vanilla SL.After server-side processing, the client completes the forward pass through its tail and computes task loss.Source: A. Design Rationale · p. 4

B. Client-Side Projection

  • The client applies a fixed, data-independent orthonormal projection, reducing each sample's transmitted representation from d scalars to k without adding learnable parameters.The matrix is sampled from a Gaussian matrix via thin QR decomposition, broadcast to clients, and held fixed; feature maps can also be projected tile by tile.Source: B. Client-Side Projection · pp. 4–5
  • The data-processing inequality gives I(x; ˜z) ≤ I(x; z), but projection does not eliminate leakage; Within-Class Compaction further suppresses exploitable sample-specific variation.Because x → z → ˜z forms a Markov chain, the projection cannot introduce information about x absent from z.Source: B. Client-Side Projection · p. 5

C. Server-Side Liftback

LightSplit’s server-side lift-back restores projected activations to dimensions compatible with the existing backbone through either a fixed linear map or a learned transformation. The two modes use the same client projection and payload but trade minimal overhead for adaptability.

C. Server-Side Liftback

  • LightSplit lifts each projected k-dimensional activation to a d-dimensional backbone input using either LIGHTSPLIT-F or LIGHTSPLIT-L.Both modes preserve compatibility with the backbone without modifying the underlying DNN.Source: C. Server-Side Liftback · p. 5
  • LIGHTSPLIT-F reuses R to map the projection back as Pz, the best Euclidean approximation within range(R), with the discarded residual in P’s null space.This lift-back adds no trainable server parameters and uses a single fixed matrix multiplication; the same R also maps the returned gradient to the client’s coordinates.Source: C. Server-Side Liftback · p. 5
  • LIGHTSPLIT-L jointly trains a lift-back MLP with the backbone to re-encode surviving projected information across d dimensions, without recovering the discarded residual.Its adaptable mapping can absorb the global re-mapping that may conflict with convolutional backbones’ spatial inductive biases, and it is the only adaptable server component when the backbone is frozen.Source: C. Server-Side Liftback · pp. 5–6
  • LIGHTSPLIT-F and LIGHTSPLIT-L send the same payload, so client cost, bandwidth, adversary view, and privacy properties are identical across modes.LIGHTSPLIT-F suits backbones with spare capacity; LIGHTSPLIT-L is suited to frozen backbones or inductive-bias mismatch, and performs better on more complex benchmarks while matching F on remaining benchmarks.Source: C. Server-Side Liftback · pp. 5–6

D. Privacy Role of Within-Class Compaction

Projection alone can retain instance-specific cues that enable reconstruction, so Within-Class Compaction (WCC) suppresses residual within-class variation while preserving class-discriminative structure. WCC’s small recommended weight captures privacy gains at negligible utility cost.

D. Privacy Role of Within-Class Compaction

  • Projection alone offers limited privacy because a server with auxiliary data and the known projection matrix can lift representations and exploit residual structure for reconstruction.Dimensionality reduction does not eliminate all instance-specific variation in transmitted representations.Source: D. Privacy Role of Within-Class… · p. 6
  • WCC minimizes within-class scatter as a surrogate for reducing I(x; ˜z | y), compacting same-class samples while preserving class-discriminative I(y; ˜z).This makes representations behave more like class prototypes than instance descriptors, reducing exploitable per-sample signals.Source: D. Privacy Role of Within-Class… · pp. 6–7
  • WCC is computed on the client using labels that never leave the client, and it adds no communication.The method uses a variance penalty rather than an adversarial loss or privacy discriminator.Source: D. Privacy Role of Within-Class… · pp. 6–7
  • λWCC≈10−3 captures the privacy gain at negligible utility cost; any λWCC≥10−3 holds the privacy floor at a small additional utility cost.Setting λWCC=0 recovers projection-only training.Source: D. Privacy Role of Within-Class… · p. 6

V. EVALUATION … 2) Evaluated Attacks:

The evaluation varies client-head depth and ownership, tests LightSplit across image, tabular, and language tasks against cut-layer baselines, and probes privacy under passive, active, and payload-based attacks. It reports task utility, compression, and reconstruction metrics.

A. Experimental Setup

  • Experiments vary image-model cut depth (L1H or L2H) and client-head ownership (per-client PCH or shared SCH), while tabular and text models use one cut.The default protocol is U-shaped split learning (USL).Source: A. Experimental Setup · p. 7

1) Datasets, models, and cut layers:

  • Utility and attack evaluations span CIFAR-10, CIFAR-100, GTSRB, MNIST, Fashion-MNIST, ImageNet-1k, UCI HAR, and GPT-2 fine-tuned on WikiText-103.The passage names MNIST and Fashion-MNIST for per-sample inversion attacks and describes 8–32× evaluations for the additional ImageNet, tabular, and language settings.Source: 1) Datasets, models, and cut layers · p. 7
  • Baselines are unmodified RAW activations, a task-trained LEARNED 1×1 channel bottleneck, and payload-matched top-k sparsification, with differential privacy tested alone and with LightSplit.Top-k transmits selected un-mixed activation coordinates; RAW is the uncompressed upper bound on cut-layer leakage.Source: 1) Datasets, models, and cut layers · p. 7
  • Compression ratio is CR = d/k, where d is the original activation dimension and k is the transmitted dimension; for RAW, LEARNED 1×1, and LightSplit, it reflects bandwidth and information.Fewer transmitted features mean less information is available at the cut layer.Source: 1) Datasets, models, and cut layers · p. 7

2) Evaluated Attacks:

  • Published passive attacks SDAR, UnSplit, and FORA retain their authors’ configurations, with LightSplit inserted after the first client-server cut and LIGHTSPLIT-F evaluated through lift-back.The reported attack tables and figures use LIGHTSPLIT-F; SDAR also tests LIGHTSPLIT-L directly.Source: 2) Evaluated Attacks · pp. 7–8
  • Additional privacy probes include active FSHA, decoders using payload z with lift-back, gradients, or their fusion, and UCI HAR reconstruction and re-identification attacks.The paper also tests BadNets trigger and label-flip poisoning setups for per-client anomaly scores.Source: 2) Evaluated Attacks · p. 8
  • Utility is measured by top-1 test accuracy, while image reconstruction is assessed with MSE, PSNR, SSIM, and LPIPS, with arrows indicating stronger privacy.Accuracy changes marked ∆ are relative to the matched RAW baseline, reported in percentage points unless explicitly stated as a relative percentage change.Source: 2) Evaluated Attacks · p. 8

B. Experimental Results … 4) Privacy Attacks:

Across image, tabular, and language-model tasks, LightSplit combines substantial compression with generally preserved utility, while its projections reduce information available to reconstruction and other privacy attacks. Its strongest privacy gains often come from pairing projection with WCC, and results also examine utility trade-offs, malicious attackers, and the learned lift-back.

1) Main Comparison with Raw, Learned, and Top-k Baselines:

  • LightSplit reduces image-attack leakage more than learned 1×1 or top-K baselines: at 8×/16×/32×, SDAR SSIM falls 37.0%/44.2%/58.3%, while FORA LPIPS rises +218%/+213%/+212%.Against UnSplit at the leakiest split-2 cut, MSE rises by +27% on MNIST and +8–9% on Fashion-MNIST; utility costs range from −0.5% to −13.7% across the reported image datasets.Source: 1) Main Comparison with Raw, Learned,… · p. 8

2) Utility Behavior:

  • On CIFAR-100, LIGHTSPLIT-L stays within ≤7 pp of RAW at 32× under Non-IID, while learned 1×1 loses ∼9.6 pp; tiled projections narrow the PCH gap to RAW from 10.8 to 4.7 points.The PCH/SCH accuracy ratios on CIFAR-100 are 0.67 for LIGHTSPLIT-L and 0.71 for LIGHTSPLIT-F; the text also reports that LIGHTSPLIT-L performs better on harder semantic tasks and LIGHTSPLIT-F is more robust on simpler datasets under Non-IID.Source: 2) Utility Behavior · pp. 8–9

3) Beyond Image Classifiers:

4) Privacy Attacks:

  • With SDAR at CR 8×, LIGHTSPLIT-F reduces SSIM by 37.0% and raises LPIPS by +61.3% without WCC; at λWCC=10−1, SSIM drops 36.7% at 8× and 105.3% at 32×.WCC can make reconstructions visually unidentifiable even when pixel metrics do not improve monotonically; across the USL sweep, WCC and high compression jointly produce stronger gains at L=4, while top-K trails every method at L=7.Source: 4) Privacy Attacks · pp. 9–10, Figure 5 · p. 10
  • Against FORA’s pseudo decoder at 8×, LIGHTSPLIT-F raises LPIPS by +217.8% and reduces SSIM by 96.0%, versus +97.7% LPIPS for learned 1×1 and +78.9% for top-K.WCC changes pseudo-decoder LPIPS only from +217.8% to +221.0% at λWCC=10−1, but raises the white-box reference result from ×3.8 to ×4.9.Source: 4) Privacy Attacks · p. 10, Figure 6 · p. 11
  • For UnSplit at split-2, LIGHTSPLIT-F raises foreground-masked MSE by +27% on MNIST and +8% to +11% on F-MNIST, while on HAR it cuts subject re-identification by 55.0% at 8×.On HAR, with λWCC=10−1, the median per-feature R2 drop is 26.4% at 8× and subject re-identification drops 61.4% at 32×; the reconstruction columns are uniformly black across datasets and splits.Source: 4) Privacy Attacks · p. 10
  • Under fully malicious FSHA, LIGHTSPLIT-F raises reconstruction LPIPS to 8.4× the no-defense value on MNIST and 2.5× on Fashion-MNIST at unchanged accuracy; learned 1×1 and top-K leave the attacker unimpeded.Attackers given the analytic lift-back gain at most +0.032 SSIM, and combining the cut representation, returned gradient, and lift-back does not outperform it; projection removes 89.8/95.6/98.5% of activation energy at 8×/16×/32×.Source: 4) Privacy Attacks · pp. 10–11
  • At the L=7 cut, LIGHTSPLIT-L SDAR SSIM is 0.18/0.12/0.14 at 8×/16×/32×, versus 0.36/0.17/0.08 for LIGHTSPLIT-F and 0.57 without defense.Both variants send the identical projected payload, and the learned lift-back is server-side post-processing on that released vector.Source: 4) Privacy Attacks · p. 11

5) Differential Privacy: … A. Security Considerations

LightSplit combines projection, WCC, and optionally differential privacy to limit reconstruction leakage while retaining utility, but its protection is scoped to passive input reconstruction and leaves class-level information exposed. The discussion also reports that backdoor signals generally remain detectable and gives deployment guidance for choosing projection variants.

5) Differential Privacy:

  • At the default L=4 cut, LIGHTSPLIT-F +DP yields the largest SDAR SSIM drop (−64.0%), versus DP alone (−41.6%) and LIGHTSPLIT-F alone (−37.0%).The chosen σ=0.02 keeps ≈97% of un-noised accuracy; at L=7, SSIM falls from 0.572 to 0.139 with both defenses, compared with 0.360 under LIGHTSPLIT-F alone and 0.181 under DP alone.Source: 5) Differential Privacy · p. 11
  • At 8–32× compression, CIFAR-10 retains above 93% and CIFAR-100 above 85% of uncompressed accuracy; at 4096×, they retain 55% and 12%, respectively.Accuracy declines earlier for CIFAR-100: at 512× it retains 63%, compared with 89% for CIFAR-10.Source: 5) Differential Privacy · p. 11

6) Backdoor Detection Signal:

  • Across most backdoor-sweep settings, RAW, LEARNED 1×1, LIGHTSPLIT-F, and LIGHTSPLIT-L retain poisoned-client identifiability above the |MAD-z| 3σ threshold.Detection weakens for strongly non-IID data (α ≤1) and drops slightly in a few client-number and poison-ratio settings.Source: 6) Backdoor Detection Signal · p. 11

8) Deployment Guidance:

  • For shared heads, global projection with WCC is preferable; for per-client heads, use block-local projection with DP.With per-client heads, tiling narrows the RAW gap from 10.8 to 4.7 points, and DP restores privacy at each tile size with little utility cost. Appendix J covers ablations of WCC strength, client-count scaling, and LIGHTSPLIT-L lift-back width.Source: 8) Deployment Guidance · pp. 11–12, 7) Ablations · p. 11

VI. DISCUSSION

  • LightSplit’s privacy gains arise from rank-reducing information loss and WCC, which suppress instance-specific signals while retaining task-relevant information.The discussion notes that LightSplit lacks an inherent formal guarantee, but can be combined with differential privacy for formal privacy gains.Source: A. Security Considerations · p. 12

A. Security Considerations

  • Clients transmit only projected representations, irreversibly discarding information; WCC further suppresses exploitable intra-class variation, while server-side lift-back preserves compatibility with existing SL architectures.Reducing transmitted activations and returned gradients from d to k lowers communication overhead with only negligible client-side computation.Source: A. Security Considerations · p. 12
  • LightSplit targets passive input reconstruction, not label leakage or gradient-channel poisoning; under FSHA, LIGHTSPLIT-F raises reconstruction error to 8.4× on MNIST and 2.5× on FashionMNIST at unchanged accuracy.Label leakage attacks exploit gradients or similarity-based clustering, while active servers deviate from protocol; learned 1×1 convolution and top-k sparse leave FSHA unimpeded.Source: A. Security Considerations · p. 12
  • The server can still recover class-level prototypes from smashed data, which the paper treats as LightSplit’s privacy floor rather than information about specific inputs.Model poisoning and backdoor injection through the gradient channel remain out of scope; the paper calls for more comprehensive evaluation of backdoor attacks.Source: A. Security Considerations · p. 12

B. Limitations … E. Positioning Our Approach

LightSplit uses a fixed, non-trainable orthogonal projection at the split layer to compress per-sample activations without client-side trainable parameters, while limiting information available to reconstruction attackers. The related work distinguishes this joint compression-and-privacy approach from prior defenses and compression methods, and notes that it does not provide formal privacy guarantees.

B. Limitations

  • LightSplit addresses smashed-data privacy against a curious server, not shared-model-update inference attacks, and plain LightSplit offers no formal guarantees.The fixed rank-deficient view reduces activation detail, while JL explains preservation of task-relevant structure; combining with Differential-Privacy [1] is described as a route to formal guarantees. LIGHTSPLIT-L adds server-side MLP parameters and compute, whereas parameter-free LIGHTSPLIT-F is an alternative with a tolerable performance gap.Source: B. Limitations · pp. 12–13

A. Privacy Threats at the Split Interface

  • The threat focus is training-time server observation of smashed data; deployment-time inference attacks, and label leakage [30], are out of scope.Active FSHA [38] manipulates client gradients, but such deviations are detectable,; it is retained as a stress test, while passive reconstruction is the primary practical threat.Source: A. Privacy Threats at the Split… · p. 13
  • Passive reconstruction includes gradient inversion and surrogate attacks; SDAR matches active FSHA [38] reconstruction quality while remaining passive.SDAR trains a simulator on auxiliary data with the server backbone frozen, exploiting distributional information retained through co-training; compression that preserves task structure may retain signal for surrogate attacks.Source: A. Privacy Threats at the Split… · p. 13

B. Existing Defenses

  • Existing defenses either leave communication payload unchanged, expose structural information, or incur substantial overhead; Censor [58]’s candidate selection is infeasible for split learning.Representation perturbation methods can be defeated by adaptive attackers. FedVS [31] uses secret sharing, while homomorphic encryption enables ciphertext computation at orders-of-magnitude overhead unsuitable for edge deployment.Source: B. Existing Defenses · p. 13

C. Communication Compression at the Cut Layer

  • Cut-layer compression alternatives expose active positions, add client-side learned-codec costs, or fail to reduce per-sample payload.Top-k variants reveal selected positions [61], [62]; learned codecs such as BottleNet++ [42] and FrankenSplit [15] add client parameters and compute; SplitFC still reveals kept positions, while C3-SL [24] only reduces transmissions per round.Source: C. Communication Compression at the… · p. 13

D. Random Projections in Distributed Learning

  • Prior random projections target other objects, and the authors state that no prior work applies one at the split-learning cut layer for per-sample activation compression.Examples include pretrained features for continual learning [33], local model parameters for personalized FL, and backward gradients for label protection in vertical FL.Source: D. Random Projections in Distributed… · p. 13

E. Positioning Our Approach

  • The stated gap is that no existing method jointly compresses, avoids client overhead, and restructures the server’s observation under a semi-honest threat model.LightSplit’s privacy benefit is evaluated empirically under state-of-the-art attacks with full knowledge of R; formal JL results require added noise [3], [26], which plain LightSplit does not add.Source: E. Positioning Our Approach · pp. 13–14
  • LightSplit projects d-dimensional activations to k dimensions, reducing per-sample payload by d/k without trainable client parameters; server-side lifting or learning and optional WCC are available.LIGHTSPLIT-F lifts with R⊤, while LIGHTSPLIT-L applies a small learned module; WCC pulls same-class activations toward shared centroids to suppress per-sample signals while preserving class separability.Source: E. Positioning Our Approach · pp. 13–14

APPENDIX … 2) FORA:

The appendix specifies model partitions and experimental setups for utility and attack evaluations, including compression, cut width, and differential-privacy sweeps. It also describes how FORA metrics and reference-decoder results are interpreted.

2) ImageNet-1k; 6) Compression range and cut width:

  • Experiments split a ResNet-50 across 10 IID clients and compare RAW, learned 1×1 convolution, top-K, and LightSplit at compression ratios 8×, 16×, and 32×.The cut dimension is d=4096; LightSplit also sweeps tile sizes b ∈ {1, 2, 4, 8}.Source: 2) ImageNet-1k · p. 15

A. Model partitions; B. Utility experiments; 1) Baselines and compression ratios:

3) Block-local projection:

  • Block-local projection tiles spatial cut maps into b×b patches and applies a shared fixed projection, with b=1 mixing channels per position and b=8 recovering the global projection.For T tiles, the projection matrix is T^2 times smaller at the same compression ratio.Source: 3) Block-local projection · p. 16

4) Tabular data (UCI HAR):

  • UCI HAR experiments use a d=256 cut and transmit k ∈ {8, 16, 32} scalars across five clients, with heterogeneous label shares drawn from a Dirichlet(α) distribution.The study uses the official subject-disjoint split, three seeds, and WCC arms with λWCC=10^-1.Source: 4) Tabular data (UCI HAR) · p. 16

5) Language modelling (GPT-2):

  • GPT-2 fine-tuning covers four model sizes from 124M to 1.5B, projects the residual state at a cut after about one-tenth of model depth, and evaluates perplexity inflation over RAW.The sweep uses compression ratios 8×, 16×, and 32× on WikiText-103; at 8×, payload dimensions are k=96 for 124M and k=200 for 1.5B.Source: 5) Language modelling (GPT-2) · p. 16

7) Differential privacy:

  • At σ=0.02, differential privacy keeps ≈97% of the un-noised accuracy on CIFAR-10 (L2H).Clients clip each per-sample payload and add calibrated Gaussian noise during training and evaluation, making noise part of the released representation.Source: 7) Differential privacy · p. 16

1) SDAR:

  • SDAR is run under both U-shaped and vanilla SL protocols, with the same cut and methods evaluated across split points and compression ratios.The block-local variant is also evaluated under SDAR, FORA, UnSplit, and FSHA; SDAR alone sweeps λWCC because its decoder co-trains with the victim.Source: 1) SDAR · pp. 16–17

2) FORA:

  • FORA uses LPIPS as its primary metric because pseudo-decoder RGB drift makes SSIM noisier; Tables II and III report SSIM as multi-seed means.Table XIII includes a white-box reference decoder trained on victim (z, x) pairs as a leakage ceiling; it is non-deployable in U-shaped SL.Source: 2) FORA · p. 17

3) UnSplit: … 2) Block-local projection:

Block-local projections narrow the average utility gap to RAW at smaller tiles, especially with per-client heads, but their interaction with WCC depends on its weight. The accompanying attack evaluations specify how reconstruction, tabular leakage, and backdoor detection are measured and configured.

3) UnSplit:; D. Reconstruction metrics

  • UnSplit uses the released pipeline and unchanged MnistNet at cuts ℓ=2 (d=1152) and ℓ=4 (d=1024), with LightSplit inserted between the client head and server backbone.Each dataset, method, and cut is trained for 20 epochs with Adam (lr 10−3, AMSGrad, batch 64); LightSplit uses LCE + λwccLWCC and ranks set by compression ratios 8, 16, and 32. MSE and PSNR use foreground pixels above 0.1, while SSIM and LPIPS use a tight crop around that mask; MSE/LPIPS ↑ and SSIM/PSNR ↓ indicate weaker leakage. PSNR is −10 log10(MSE) for images scaled to [0, 1].Source: 3) UnSplit · p. 17, D. Reconstruction metrics · p. 18

4) Tabular probes:

  • The tabular probes invert the 256-d HAR cut using a decoder evaluated on members and a subject classifier evaluated against the 1/|subjects| chance rate.Features are standardized so mean prediction scores R2=0; group-aware splits and activity-stratified member/non-member samples are used, with a held-out member slice selecting the checkpoint.Source: 4) Tabular probes · p. 17

5) Active attack: FSHA:

  • FSHA is run with its released training loop, WGAN objective, and learning rates, with LightSplit inserted at the client-encoder output so the attack sees the transmitted payload.For LIGHTSPLIT-F, the attacker’s decoder and discriminator are fully connected to match the projected k-vector; runs use 10,000 iterations and repeat reconstruction cells over several seeds, scoring 20 images per seed and reporting the mean.Source: 5) Active attack: FSHA · p. 17

6) Projection-optimized attackers:; 7) Differential privacy under each attack:

  • The projection-optimized attackers differ in whether their decoders consume z alone, its analytic lift-back, the returned cut-layer gradient, or all three fused.The z-only decoder is widened to match the lift-back decoder’s parameter count. DP uses σ=0.02 and a per-configuration median payload-norm clip; the DP mechanism is placed at the cut, with the same recipe used for block-local rows.Source: 6) Projection-optimized attackers · p. 18, 7) Differential privacy under each… · p. 18

8) Backdoor detection:

  • Backdoor detection uses a 3×3 corner trigger and label flip in multi-client CIFAR-10 split learning, with clients flagged by cosine-to-consensus signatures against a 3σ-MAD threshold.Defaults are 10 clients, Dirichlet α=107, malicious fraction 0.1, poison ratio 0.3, target class 0, and compression ratio 16; one factor varies at a time. The study retains the standard detector and does not consider more advanced attack or defense strategies.Source: 8) Backdoor detection · p. 18

1) Full utility baseline experiments:

  • Table VII gives best test accuracy for every dataset, distribution, head ownership, head depth, and compression-ratio cell, with changes measured against matched RAW baselines.The four head configurations cross L1H/L2H with SCH/PCH; LIGHTSPLIT-L uses per-dataset widths.Source: 1) Full utility baseline experiments · p. 18, Table VII · p. 19

2) Block-local projection:; 9) Center loss vs. WCC:

  • At λWCC=0, b=2 improves every LIGHTSPLIT-F configuration, changing L2H-PCH’s loss from −9.1% to +0.2% and L1H-SCH’s from −9.5% to −3.8%.As λWCC grows, the advantage reverses: at 10−2, L2H-PCH with a tile falls to −58.2%, versus −33.0% with global projection. A smaller tile leaves WCC a smaller subspace to whiten; center loss was also tested against WCC on the same SDAR setup, but no outcome is supplied here.Source: 9) Center loss vs. WCC · p. 18, 2) Block-local projection · pp. 19–20

2) Block-local projection:

  • LIGHTSPLIT-F’s average accuracy gap to RAW is 4.5 points at b=1, 5.0 at b=2, 7.4 at b=4, and 9.2 at global b=8.At matched bytes, the per-client-head gap falls from 10.8 points at b=8 to 3.1 at b=1 and 4.7 at b=2; shared-head gaps move from 7.6 to 5.9 and 5.3. LIGHTSPLIT-L has gaps of 11.1, 10.0, and 9.4 points at b=1, 2, and 4, respectively.Source: 2) Block-local projection · pp. 18–19, Table VIII · p. 20

3) ImageNet-1k: … 1) SDAR:

The merged results examine utility across ImageNet-1k, UCI HAR, and GPT-2, alongside SDAR reconstruction privacy. They show how utility and privacy outcomes vary with compression and experimental settings.

3) ImageNet-1k:

  • Fig. 7 evaluates ImageNet-1k top-1 accuracy at 8×, 16×, and 32×, sweeping tile sizes and comparing the b=2 operating point with matched-budget baselines.The figure covers 1000 classes using ResNet and an 8×8×64 cut with a shared deep head; the supplied passages do not report its accuracy values.Source: 3) ImageNet-1k · p. 20, Figure 7 · p. 20

4) Tabular (HAR): full compression sweeps:

  • LIGHTSPLIT-F stays within 1.5 points of RAW at every compression ratio and heterogeneity level, while trained bottlenecks lose more as clients become more heterogeneous.At α=0.3, LEARNED 1×1 loses 0.6–0.9 points; at α=0.1, it loses 5.7–10.9 points. LIGHTSPLIT-L follows the same trend more mildly. The fixed projection has no trainable parameters, so client data skew cannot bias it.Source: 4) Tabular (HAR): full compression… · p. 20, Table IX · p. 21
  • Top-K sparse stays within 1.7 points of RAW at 8–16× for every α, but loses about 3 points at 32× when α ≤0.3.At α=0.5, all arms match the IID result: RAW is 94.17% against 94.01% IID. The arms separate only from α ≤0.3; sparse selection can discard coordinates needed by classes absent from a client.Source: 4) Tabular (HAR): full compression… · p. 21, Figure 8 · p. 21

5) Language modelling:

  • At CR 8×, GPT-2 validation-perplexity inflation falls from 1.15× on the 124M model to 1.06× on the 1.5B model; on 124M it rises to 1.36× at 32×.The CR 8×–32× inflation gap shrinks from 0.21 to 0.05 across those model sizes. The passages attribute the size trend to wider residual states carrying more redundancy; LIGHTSPLIT-L is 1.18× against LIGHTSPLIT-F at 1.15× on 124M at CR 8×, narrowing to 1.09× against 1.06× by 1.5B.Source: 5) Language modelling · p. 21, Figure 9 · p. 21

F. Privacy Attacks

  • The privacy-attack section reports the full grid for each attack in main-text order.The supplied passage identifies the section’s reporting scope but does not state results for attacks beyond SDAR.Source: F. Privacy Attacks · p. 21

1) SDAR:

  • At CR 8×, LIGHTSPLIT-F lowers VSL SDAR SSIM from 0.565 to 0.342 at L=7 and from 0.794 to 0.495 at L=4; WCC lowers these to 0.100 and 0.081.Without WCC, these results are close to the corresponding USL values of 0.360 and 0.524, indicating the privacy gain does not depend on protocol.Source: 1) SDAR · p. 21, Table X · p. 22, Figure 10 · p. 23
  • Across twelve SDAR settings, stronger WCC moves SSIM and PSNR toward stronger privacy in nine cells, LPIPS in nine, and all three metrics together in seven.The sweep is not monotone everywhere; one exception is at L=7, 32×, where SSIM is already about 0.08 without WCC and shifts by at most 0.04. At USL L=4, CR 8×, SSIM changes from 0.524 to 0.374 and then 0.527, while LPIPS rises from 0.468 to 0.497 to 0.533. WCC can yield class-typical rather than sample-specific images, so pixel metrics and LPIPS can diverge.Source: 1) SDAR · pp. 21–22
  • At the 8192-d cut, SDAR reconstruction depends on tile size: without WCC, b=8 is 42–57% of RAW SSIM, while small tiles are 71–86%.At the 4096-d cut, projection alone holds every tile at 16–56% of RAW, with no tile-size ordering. At 8192-d, WCC lowers wider tiles but never brings b=1 below 64%; DP at 8× brings 4096-d tiles to 23–26% of RAW and lowers every 8192-d tile. Each table cell is a seed mean; above λWCC=10−2, values vary strongly between seeds.Source: 1) SDAR · pp. 22–24, Table XII · p. 23

2) FORA: … G. Differential Privacy

Across reconstruction attacks, LightSplit’s projected representations limit recovery, while block-local variants, projection-aware attackers, and differential privacy reveal how attack strength and utility change with the setting.

3) UnSplit:

  • On UnSplit’s block-local variant, whole-image SSIM reaches at most 0.229 on MNIST and 0.073 on Fashion-MNIST across tile sizes.With DP at 8×, tile SSIM is 0.114 to 0.127 on MNIST and 0.043 to 0.046 on Fashion-MNIST, matching the global projection’s level.Source: 3) UnSplit · p. 24

4) Tabular probes:

  • Across HAR compression ratios, both probe curves fall as compression tightens, and WCC lowers feature-reconstruction and subject-re-identification results at every ratio.Top-K sparse’s low R2 reflects unrecoverable zeroed coordinates, while its re-identification accuracy matches the other methods, so the R2 dip is not a privacy gain.Source: 4) Tabular probes · p. 24
  • Under malicious FSHA at CR=8×, LIGHTSPLIT-F makes MNIST and Fashion-MNIST unrecognizable; WCC widens ∆Raw to ×18 and ×5.5, while +DP reaches ×16.7/×3.4 and MNIST SSIM 0.342 with accuracy unchanged.RAW, the learned 1×1 convolution, and top-K sparse remain legible under the same attack.Source: 4) Tabular probes · p. 24
  • For block-local FSHA, b=1 tiles at 16×/32× reconstruct nearly as well as RAW, while DP lowers b=2 and b=4 SSIM but raises b=1 above RAW.On MNIST, b=1 yields 0.65/0.64 versus RAW’s 0.68; on Fashion-MNIST it yields 0.56/0.54 versus RAW’s 0.61.Source: 4) Tabular probes · p. 24

6) Projection-optimized attackers:

  • For projection-optimized attackers, lift-back raises SSIM over reading z alone by +0.032, +0.020, and +0.015 at CR 8/16/32×, while adding the returned gradient g gives no gain.The attacker concatenating z, g, and zR⊤ remains below the lift-back decoder and even below z alone at every ratio.Source: 6) Projection-optimized attackers · p. 24

7) Energy discarded by the projection:

  • The share of cut-layer activation energy lost after projection and lift-back grows with compression, and that discarded portion never leaves the client.Thus, it remains out of reach of the attackers regardless of which signals their decoders read.Source: 7) Energy discarded by the projection · p. 24

2) FORA:

  • At λWCC=0, LIGHTSPLIT-F’s ∆Raw is ×3.8/×4.4/×4.8 across 8×/16×/32×, exceeding LEARNED 1×1 (×1.9/×2.5/×3.4) and top-K sparse (×2.0/×3.0/×3.6).The pseudo decoder’s SSIM stays at or below 0.12 for LIGHTSPLIT-F at every ratio and λWCC; at λWCC=10−1, LIGHTSPLIT-F’s ∆Raw reaches ×4.9/×5.4/×5.6.Source: 2) FORA · p. 24, Table XIII · p. 25

G. Differential Privacy

  • With DP, LIGHTSPLIT-F on GTSRB IID records 95.6/94.0/90.6% accuracy at 8×/16×/32×, with costs within ∼2 points on stable datasets and no increase with compression.Block-local DP costs stay within ∼1 point across b ∈{1, 2, 4}. Lift-back is post-processing of the privatized payload; at σ=0.25 the projection reaches 68.0% against the raw cut’s 52.6%, while σ=0.02 is described as nearly free.Source: G. Differential Privacy · p. 25

H. Compression range and cut width · I. Backdoor detection: signal sweeps and F1 table · J. Ablations

Across compression and cut-width sweeps, LightSplit balances utility and reconstruction resistance; it also preserves backdoor-detection signals, while ablations identify effective settings and trade-offs for regularization and lift-back design.

H. Compression range and cut width

  • Within the 8–32× band, each doubling costs CIFAR-10 under one accuracy point; CIFAR-100 stays above half its uncompressed accuracy through 512× compression.CIFAR-10 stays above half at every tested ratio, while both datasets’ accuracy curves fall only when the payload shrinks to a handful of coordinates.Source: H. Compression range and cut width · p. 25, Figure 15 · p. 29
  • At d=4096, projected SDAR SSIM falls from 0.354 at CR 8× to 0.179 at CR 32×, below the weakest RAW cell, 0.457 at d=512.At matched payloads, widening the cut improves accuracy by +3.9, +3.3, and +3.5 points for k=64, 128, and 256, respectively; the wider cut also costs the client more parameters, so d=4096 is retained.Source: H. Compression range and cut width · pp. 25–26, Figure 16 · p. 29

I. Backdoor detection: signal sweeps and F1 table

  • F1 stays in [0.67, 1.00] across the operative regime, reaching 1.0 in most cells, while LightSplit preserves or amplifies malicious-client signals relative to RAW.The regime is α ≥10, p ≤0.3, every tested µ, and every tested n; zero-F1 cells fail the strict 3σ threshold even though the malicious client remains the most anomalous.Source: §I · p. 26, Figure 17 · p. 29, Figure 18 · p. 29

J. Ablations

  • With a shared L2H-SCH head, λWCC≈10−3 is recommended: accuracy is 83.3% at 10−3 versus 82.8% at 0 and 81.6/80.7% at 10−2/10−1.Reconstruction is already at its floor at 10−3; with λWCC=0, projection alone lowers SDAR SSIM to 0.34/0.16/0.20 at L7 and 0.52/0.46/0.35 at L4 for CR 8/16/32×. For λWCC ≥10−3, the L7 attacker stays at 0.09–0.13 SSIM across ratios.Source: J. Ablations · pp. 27, 29, Table XXI · p. 31, Figure 19 · p. 31
  • For per-client heads, WCC is costly: LIGHTSPLIT-F L2H-PCH loses 16.4% versus RAW at 10−3 and 46.9% at 10−1, so the paper recommends no WCC, block-local projection, and DP.At 8× on CIFAR-10, L2H-PCH remains within 2.7 points of RAW with b=2 tiles versus 5.7 points for global projection; DP cuts SDAR SSIM from 0.360 to 0.139 at L7, at an average +0.7-point cost over LIGHTSPLIT-F PCH cells.Source: J. Ablations · pp. 30–31
  • At matched strengths, center loss and WCC have similar reconstruction privacy, so the stateless per-batch WCC is retained over stateful centers.Median SDAR SSIM is 0.09 for center loss versus 0.10 for WCC at λ=0.1, and 0.11 versus 0.09 at λ=0.01; center loss adds no measurable privacy gain and requires persistent centers and synchronization.Source: J. Ablations · p. 31, Figure 21 · p. 32
  • The LIGHTSPLIT-L lift-back’s best hidden width varies by dataset, while increasing lift-back size helps CIFAR-100 more than the other tested datasets.For L2H-SCH, best settings are h=512 on CIFAR-10 (85.91, −1.33 vs. RAW), h=2048 on CIFAR-100 (55.34), h=128 on FashionMNIST (91.22), and h=128 on GTSRB (95.88). On CIFAR-100, scaling ϕ from 1.2M to 9.4M tightens the accuracy gap by 3.11 pp (−7.24→−4.13); on FashionMNIST, GTSRB, and CIFAR-10 the best-to-worst spread is ≤1.98 pp.Source: Table XIX · p. 31, J. Ablations · p. 32, Figure 22 · p. 32
  • As client count rises from 10 to 100, LIGHTSPLIT-F loses at most 0.67 pp on CIFAR-10 and 3.33 pp on CIFAR-100; LIGHTSPLIT-L loses at most 0.25 and 2.60 pp, respectively.On CIFAR-100, the gap to RAW narrows as client count grows because RAW accuracy drops 5.08 pp while LIGHTSPLIT-L drops 2.60 pp.Source: Table XX · p. 31, J. Ablations · p. 32

K. Runtime Evaluation on Representative Hardware · L. Computational complexity · M. Training steps and WCC backprop

LightSplit adds modest runtime overhead on representative edge hardware while reducing cut-interface communication, with fixed and learned lift-back variants trading simplicity for server-side adaptation. Its training setup keeps the projection fixed and leaves the client-server protocol unchanged when WCC is enabled.

K. Runtime Evaluation on Representative Hardware

  • At 32× compression, average runtime is 0.204 s per batch, an 8.1% overhead over the uncompressed 0.189 s baseline.At 8× and 16× compression, runtime is 0.247 s and 0.216 s, with overheads of approximately 30.8% and 14.3%, respectively.Source: K. Runtime Evaluation on… · p. 32
  • At 32× compression, per-sample amortized runtime increases from 1.48 ms to 1.60 ms; the client stores R in 1.0 MB of fp16 at setup.At 8× and 16× compression, R requires 4.2 MB and 2.1 MB of fp16 storage, respectively, or can be regenerated from its seed.Source: K. Runtime Evaluation on… · p. 32

L. Computational complexity

  • Both LightSplit variants send bk scalars per batch; LIGHTSPLIT-F adds O(bdk) work per side and no trainable parameters, while LIGHTSPLIT-L adds O(m(k + d)) server parameters and O(bm(k + d)) work.The added costs are measured beyond the common split model h◦g◦f.Source: L. Computational complexity · p. 33
  • WCC adds O(bk) client arithmetic only during training, sends no extra values, and disappears at inference.Source: L. Computational complexity · p. 33

M. Training steps and WCC backprop

  • In the two-optimizer training setup, the client optimizes f and h while the server optimizes g and, for LIGHTSPLIT-L, the learned liftback ϕ; fixed R is untouched by either optimizer.Source: M. Training steps and WCC backprop · p. 33
  • With or without LWCC, the cut-interface forward and backward payloads and the server parameter update are bit-identical to plain U-shaped split learning.Source: M. Training steps and WCC backprop · p. 33

1) Forward pass: … 5) Wire-level and optimizer-level summary:

The protocol sends projected cut-layer activations while keeping labels and the within-class compaction loss client-local, then combines client-local and server-propagated gradients. Its wire protocol remains standard U-shaped split learning; setting λWCC > 0 adds only a client-local term to the client’s feature-extractor update.

1) Forward pass:

  • The client computes cut-layer activations, projects them with fixed R, and sends only the projections; labels remain local as the server lifts each projection and computes its backbone output.The server returns those outputs to the client for the tail and cross-entropy computation.Source: 1) Forward pass · p. 33

2) Backward pass: client (Phase 1):

  • During client Phase 1, cross-entropy gradients reach the client head and stop at u, while LWCC gradients propagate through the projection to f; the client sends ∂LCE/∂u to the server.Labels partition the batch for LWCC and never leave the client.Source: 2) Backward pass: client (Phase 1) · p. 33

3) Backward pass: server:

  • The server backpropagates the incoming cross-entropy gradient through g and ϕ, updates its parameters using LCE alone, and returns ∂LCE/∂˜z to the client.The learned ϕ is updated only in LIGHTSPLIT-L.Source: 3) Backward pass: server · p. 33

4) Backward pass: client (Phase 2):

  • In client Phase 2, the returned ∂LCE/∂˜z joins the existing LWCC gradient, propagates through fixed R and f, and contributes to the client’s feature-extractor update alongside LWCC.The client also updates h from its cross-entropy gradient.Source: 4) Backward pass: client (Phase 2) · p. 34

5) Wire-level and optimizer-level summary:

  • Each mini-batch exchanges four cut-interface messages, one tensor per sample per direction; server and client-head updates depend only on LCE, while the client feature extractor receives LCE and LWCC contributions.The projection R is fixed and never updated.Source: 5) Wire-level and optimizer-level… · p. 34
  • Setting λWCC = 0 recovers standard U-shaped split learning, while λWCC > 0 adds only the client-local term to the feature-extractor update.Neither the cut-layer messages nor the server-side update changes form.Source: 5) Wire-level and optimizer-level… · p. 34

6) Equivalence between single- and two-optimizer implementations: · N. Compact derivation in equations

LightSplit’s single- and two-optimizer implementations produce identical parameter updates for parameter-wise first-order optimizers. The compact derivation shows how CE and WCC gradients travel through the split model, meet on the client, and recover plain U-shaped split learning when the WCC weight is zero.

6) Equivalence between single- and two-optimizer implementations:

  • For parameter-wise first-order optimizers, separate client- and server-side zero_grad, backward, and step sequences produce the same f, h, g, and ϕ updates as one optimizer sequence.This equivalence follows because each optimizer uses the parameter’s accumulated gradient, determined by the autograd graph rather than the process boundary.Source: 6) Equivalence between single- and… · p. 34

N. Compact derivation in equations

  • The derivation composes f, the fixed projection R⊤, ϕ, g, and h, with loss L = LCE + λWCC LWCC.LCE depends on the projected representation through the round-trip server path, whereas LWCC depends only on the projected representation and labels.Source: N. Compact derivation in equations · p. 34
  • The client sends the CE gradient at ui to the server; this gradient is independent of LWCC, whose partial derivatives with respect to server-side variables vanish.The loss dependencies separate the server’s CE computation from the client-side WCC contribution.Source: N. Compact derivation in equations · p. 34
  • On the client, autograd accumulates the WCC contribution, then adds the server’s returned contribution; this is where CE and WCC information meet.The client continues the fused gradient through the projection and head.Source: N. Compact derivation in equations · pp. 34–35
  • The server uses the incoming CE gradient to compute server-side gradients and returns a single backward quantity to the client.The server-side parameter gradients are gradients of the combined loss as well as LCE.Source: N. Compact derivation in equations · pp. 34–35
  • The projection matrix R is fixed and receives no optimizer update.The derivation gives SGD updates for brevity and states that other parameter-wise first-order rules apply analogously with their own state.Source: N. Compact derivation in equations · p. 35
  • Setting λWCC=0 recovers plain U-shaped split learning, with the cut-interface payloads and update rules depending only on LCE and the model.The WCC term is still computed and applied on the client.Source: N. Compact derivation in equations · p. 35

Related papers

Loading 2605.13265v2…