PaperlayerView exact revision on arXiv

Source-linked AI summary

A Privacy Study of Sparse Collaborative Inference

Maximilian Andreas Hoefler, Karsten Mueller, Wojciech Samek

arXiv:2608.16236v1cs.LG

TL;DR

Sparse collaborative inference is intended to reduce communication cost while protecting input privacy, but evidence for the privacy benefit of sparsity is limited. This paper separates sparse activations into retained values and positions, reconstructs inputs from each component, and finds that positions retain serious leakage even at low transmission rates and utility.

  • Problem

    Although sparsity is proposed to reduce both communication cost and privacy leakage in collaborative inference, the relative privacy contribution of retained values and positions remains insufficiently established.

  • Method

    The paper decomposes sparse activations into values and positions, assigns each component a bit cost, and evaluates model-only and auxiliary-data inversion attacks.

  • Results

    Sparsification reduces transmission rate faster than leakage, while positions alone remain a serious privacy risk, including above-chance re-identification even when fine visual detail is lost.

  • Takeaways & Limitations

    Audits of sparse collaborative inference should treat input-dependent positions as sensitive transmitted data rather than harmless decoding side information.

  • Takeaways & Limitations

    The experiments leave other sparsity schemes, transformer token pruning, deployed entropy codecs, and measured end-to-end wire rates for future work.

Abstract

from arXiv · show

Collaborative inference (CI) splits a model between an edge device and a server, whereby the client computes an intermediate activation, transmits it, and the server completes the computation. This raises two concerns, the communication cost of the transmission and the risk that it reveals private information about the input. Recent work reduces this cost by sparsifying activations and entropy-coding the result. Sparsity has also been argued to improve privacy, on the intuition that transmitting fewer values reveals less about the input. We test this claim by decomposing the sparse activation into the retained values and the set of positions they occupy, and by reconstructing inputs from each component in isolation. We find that sparsification reduces the leakage far less than it reduces the transmission cost, and that the remaining risk shifts to the positions, which prior analyses treat as side information for decoding. Across natural-image and face datasets, the positions alone constitute a serious privacy risk, enabling high-fidelity reconstructions and re-identification of individuals. The leakage from the positions persists even when both the transmission cost and the task utility are low. We conclude that the positions of sparse activations should be treated as sensitive transmitted data and audited carefully in the context of collaborative inference. Code is available at https://github.com/an7123/Privacy-Study-Sparse-CI.

1 Introduction

Collaborative inference reduces edge computation by transmitting intermediate activations, but those activations create communication and privacy risks. This work studies sparse activations by separating retained values from input-dependent positions and finds that positions can dominate leakage, including identity leakage, despite lower transmission rates.

  • Motivation: Collaborative inference splits computation between an edge device and server, reducing client burden while potentially increasing communication because intermediate activations can exceed the input size.The client sends an intermediate activation that the server uses to complete inference.
  • Motivation: Intermediate activations are not private by construction, and prior work has shown that transmitted features can enable input recovery.Existing defenses perturb, prune, or adversarially train transmitted representations to reduce leakage.
  • Method: Sparsification changes the adversary’s observation into retained values and retained positions, both of which can be exploited.The study reconstructs inputs from each component in isolation to identify which component carries leakage.
  • Findings: Sparsification reduces transmission rate faster than privacy leakage, including at operating points where both rate and task utility are low.The rate includes bits encoding retained positions and bits encoding retained values.
  • Findings: Positions alone can re-identify individuals far above chance even when sparsification removes fine visual detail.The study evaluates visual leakage with SSIM and biometric leakage with FaceNet similarity and rank-1/rank-5 retrieval.
  • Implications: Audits should include learned inversion attacks and treat input-dependent positions as sensitive transmitted data rather than harmless decoding side information.The auxiliary-data attacker can reveal more positional leakage than the standard white-box audit.

2 Related Work

Prior collaborative-inference work reduces feature payloads through compression and sparsification, while privacy research measures leakage through feature inversion and proposes reconstruction defenses. This study extends those lines by testing sparse activation values and positions as separate leakage channels, building on prior index–value protection and support-recovery work.

  • Feature compression: CI systems reduce transmitted-feature payloads through intermediate reduction, learned end-to-end features, or sparsification with entropy coding.Sparsification includes relevance-guided static masking with entropy coding.
  • Feature inversion: Feature inversion recovers inputs by optimizing matching images or learning inverse mappings from features, including model-inversion attacks in CI.The cited CI attacks train inverse models using auxiliary data.
  • Component-wise attacks: The study applies these two attacker families component-wise to sparse activations and measures leakage through retained values and support positions.Prior one-bit compressed sensing shows that support alone can suffice for recovery.
  • Reconstruction defenses: Reconstruction defenses include feature noise, attacker-aware training, sparsity-based obfuscation, and information-theoretic privacy bounds.A closely related method separates sparse activations into index and value channels and protects each with an information-theoretic budget.

3 Study Design

The study decomposes top-k sparse activations into input-dependent positions and retained values, then probes and inverts each component separately. It compares leakage and bit cost across representations using model-only and auxiliary-data attackers, with controls for positional structure and value-inversion architecture.

  • Activation decomposition: Top-k activations are treated as two separately transmittable components: the support positions and the non-negative retained values.All splits occur after ReLU, so retained values and magnitudes coincide.
  • Rate accounting: The study assigns separate bit costs to positions and values, enabling leakage comparisons against transmission rate.The normalized rate is H2(ρ) for positions plus ρb for values, with top-k values encoded at FP16 precision (b = 16).
  • Probes: Positions-only preserves the input-dependent mask without values, whereas values-on-random-positions preserves values while replacing positions with a fresh input-independent index set.This contrast isolates which component drives privacy leakage.
  • Controls: Dense, top-k, and quantized references bound leakage, while block-shuffled and random-mask controls test how much positional structure reconstruction requires.The block-shuffled control applies a per-sample 4 × 4 block permutation; the random control uses a matched-density input-independent mask.
  • Attackers: Two attackers invert observations: a model-only white-box optimizer and a server-side learned inverse trained with auxiliary data from the same or a related distribution.The learned inverse is especially relevant for discrete positions and can surpass the white-box attacker with as few as 100 images.
  • Value inversion controls: Permutation-invariant attackers—a sorted-value MLP and DeepSets—control for architecture mismatch when reconstructing unordered values.This ensures failed reconstruction of values-on-random-positions is not attributed solely to an unsuitable convolutional inverse.

4 Results

At 95% sparsity, input-dependent positions carry most reconstruction and identity leakage: positions-only reconstructions nearly match full top-k activations across datasets, layers, and backbones. The result persists under per-channel top-k, while attacker choice substantially affects measured leakage.

  • Evaluation metrics: The evaluation distinguishes downstream top-1 accuracy, rate in bits per activation dimension, visual leakage via SSIM, and FaceScrub biometric leakage via FaceNet cosine and held-out-gallery re-identification.The biometric protocol uses one held-out image per identity, preventing retrieval by matching the query image itself; chance rank-1 is 1/211 = 0.47%.
  • Attacker comparison: On FaceScrub, positions-only reconstructions reach 0.751 SSIM and 0.510 FaceNet cosine under Aux, whereas WB reaches only 0.076 SSIM.WB positions-only reconstructions are too degraded for meaningful identity embedding, showing representation-specific attacker differences.
  • Robustness: Positions-only leakage closely follows full top-k activation across ResNet-18 split layers 0–3 and across ResNet-18, VGG11-BN, MobileNetV3, and EfficientNet-Lite.Values on random positions remain at the floor, while usable utility appears only at deeper splits; absolute leakage differs between backbones.
  • Per-channel sparsity: Under per-channel top-k, removing channel-selection information leaves the ordering intact: positions still carry the leakage through spatial layout.Per-channel top-k fixes the number of retained units in every channel and reveals only spatial positions within each channel.

5 Discussion

Sparse activation positions are themselves sensitive input-dependent codes: positions-only attacks can reconstruct visual content and preserve facial identity, even when transmission rate and measured utility are low. Privacy evaluation should therefore test learned attackers with both visual and biometric metrics, while defenses must balance leakage against server accuracy.

  • Position leakage: Input-dependent top-k positions enable reconstructions nearly as strong as those from full sparse activations and preserve identity on faces.Exposing positions while protecting transmitted values leaves a substantial visual and biometric attack surface.
  • Position leakage: At 95% sparsity, positions-only masks send under a third of a bit per dimension yet re-identify unseen subjects at 44× chance.Leakage decreases only slowly as rate and utility drop, showing that low measured utility does not imply low information content.
  • Evaluation: SSIM rates positions-only reconstructions as poor, while FaceNet retrieval still recovers subject identity from them.Pixel similarity alone therefore understates privacy risk.
  • Evaluation: White-box audits recover little from discrete masks, whereas learned inverses recover faces, so privacy claims should use both visual and biometric metrics.The learned inverse is the stronger available attack when the server could plausibly possess auxiliary data.
  • Defenses: Freshly randomizing positions stops leakage but collapses server accuracy; fixed input-independent positions preserve server operation while transmitted values still leak.Randomized sparsification trades these effects against each other.

6 Scope and Limitations

The experiments use ResNet-18 split at layer2, with results shown to be robust across split layers and backbones. Important limitations include untested sparsity and pruning schemes, deployed entropy codecs, and analytical rather than on-the-wire rate estimates.

  • Scope: Experiments use ResNet-18 split at layer2, while results remain robust across split layers and backbones.Section 4.5 reports this robustness.
  • Limitations: Open settings include channel pruning, block and N:M sparsity, transformer token pruning, and deployed entropy codecs.Transformer token pruning is itself a positions-only transmission.
  • Limitations: Reported rates are analytical upper bounds from Eq. (3), not on-the-wire measurements.Correlated positions can compress further than the bound.

7 Conclusion

The study finds that sparsifying intermediate activations improves collaborative-inference privacy only in limited scenarios. Most remaining leakage is carried by activation positions, which form a low-rate representation of the input.

  • Sparsification reduces privacy leakage far less than it reduces transmission rate.
  • The remaining leakage is carried almost entirely by the positions of retained activation values.
  • Activation positions act as a low-rate representation of the input.

A Implementation and Evaluation Details · A.1 Split Model and Threat Model

The study splits a torchvision ResNet-18 after layer2 and evaluates sparse transmitted activations across three image and face settings. Its threat model gives white-box attackers architectural and client-side knowledge, while decoders see only the transmitted object at test time.

  • A.1 Split Model and Threat Model: The client computes the ResNet-18 stem, layer1, and layer2, while the server computes layer3, layer4, avgpool, and the final fully connected layer.The split occurs at the output of layer2.
  • A.1 Split Model and Threat Model: The transmitted activation follows layer2 ReLU and is non-negative, making retained values coincide with their magnitudes.
  • A.1 Split Model and Threat Model: The split activation has dimension d = C × H × W with C = 128 across all datasets.The dimensions are d = 8192 for TinyImageNet, d = 4608 for FaceScrub at 482, and d = 51200 for Imagenette-160.
  • A.1 Split Model and Threat Model: Top-k selection is global over the full C × H × W tensor rather than per channel or location, with ties broken deterministically by torch.topk.
  • A.1 Split Model and Threat Model: At the headline operating point ρ = 0.05, the system retains k = 409 coordinates from the split activation.The keep fraction is defined as ρ = k/d.
  • A.1 Split Model and Threat Model: Both attackers are white-box with respect to the client and know the architecture, split point, sparsification rule, and transmitted object o = O(z).
  • A.1 Split Model and Threat Model: The learned inverse trains on auxiliary-data pairs generated by querying fc, whereas the whitebox optimizer backpropagates through fc.
  • A.1 Split Model and Threat Model: At test time, both decoders observe only the transmitted object o and never the original input x or activation z.

A.2 Probes and Server Input

The probes map activation z to transmitted objects evaluated by a frozen server, including a literal 0/1 binary mask representing retained positions. Positions-only transmission loses accuracy because the mask is not magnitude-rescaled before server execution.

  • Probe design: Each probe maps activation z to a transmitted object that the frozen server f_s uses for utility measurement.The design includes m ∈ {0, 1}^d as the top-k mask and elementwise multiplication ⊙.
  • Mask transmission: The binary mask is transmitted as literal 0/1 floats occupying the activation tensor, with no magnitude rescaling.This models transmitting the index set while running the unchanged server.
  • Mask transmission: 2.4, 3.7, and 11.0 points are lost by positions-only transmission on TinyImageNet, Imagenette, and FaceScrub, respectively.The loss occurs relative to the top-k activation rather than none because the server receives an unscaled mask.

A.3 Server Classifier

The server uses the original frozen classifier without adaptation, feeding each transmitted representation directly into it to measure downstream top-1 accuracy and deployment utility.

  • Server Classifier: The server remains the original frozen fs and is not retrained, fine-tuned, calibrated, or re-normalized.This preserves the unmodified server-side classifier for evaluating each transmitted representation.
  • Server Classifier: Each transmitted object is fed directly to the frozen server, with downstream top-1 accuracy measured on this head.The evaluation therefore reflects the utility obtained from each transmitted object without classifier modification.
  • Server Classifier: The frozen-server design measures the utility that an unmodified deployment actually obtains from each transmitted object.The passage characterizes this as a deliberate design choice rather than a limitation.

A.4 Learned Inverse … A.12 Ethics and Data Governance

The paper evaluates learned, white-box, and values-only inversion attacks under matched decoder capacity, using controlled datasets and ablations to isolate leakage from sparse activation values, positions, and static supports. It also documents governance measures for the FaceScrub re-identification benchmark and treats transmitted index streams as sensitive data.

  • A.4 Learned Inverse: The learned inverse maps spatial or vector transmissions into a shared [256, 8, 8] latent, then uses an identical generative trunk to reconstruct images in [0, 1].Spatial inputs use Conv2d(C, 256, 3) →BN → ReLU, while vector inputs use a linear map followed by reshape; the trunk contains three upsampling convolutional blocks and a sigmoid output.
  • A.4 Learned Inverse: The inverse is trained with Adam, learning rate 2 × 10−3, MSE loss, batch size 256, and identical architecture, capacity, and optimization budgets across probes.Main TinyImageNet models train for 12 epochs, FaceScrub models for 20 epochs, and only the input-head channel count changes.
  • A.7 Datasets: Partitions, Preprocessing, Provenance: The evaluation uses fixed, documented datasets and partitions: TinyImageNet-200 has 100k training and 10k validation images at 64 × 64, while FaceScrub contains 526 identities and 41,425 training images.TinyImageNet evaluation uses a fixed 1000-image validation set; FaceScrub’s split classifier reaches 76.5% validation top-1 over 526 identities, with 0.19% chance.
  • A.9 Positional Structure Ablation: At fixed density, channel shuffling, spatial shuffling, and reducing supports to per-channel active counts lower leakage near the random-positions floor, showing that positional structure is required.The intact support reaches 0.428 SSIM, compared with 0.136 after channel shuffle, 0.132 after spatial shuffle, and 0.158 for per-channel counts.
  • A.10 Conditional Contribution of the Values: With the correct support, retained values add only 0.007 SSIM on TinyImageNet, 0.020 on Imagenette, and 0.4 rank-1 points on FaceScrub beyond positions-only leakage.The same values placed on random positions fall to 0.145 SSIM, indicating that positions provide most unconditional information while values can still contribute conditionally.
  • A.11 The Static Mask in Detail; A.12 Ethics and Data Governance: A static input-independent mask removes the position channel but still leaks through transmitted values, whereas values on per-sample random supports collapse to the dataset floor.FaceScrub is confined to a closed re-identification benchmark, reconstructions are retained only for evaluation, and the study treats index streams as sensitive data.
Loading 2608.16236v1…