Source-linked AI summary
CutClean: Neural Network Pruning for Privacy-Preserving Inference
Leonardo Magliolo, Vito Paolo Pastore, Giuseppe Valenzise, Enzo Tartaglione
TL;DR
Privacy leakage in neural networks remains difficult to measure and reduce without sacrificing deployment efficiency. CutClean uses privacy heads to guide structured pruning, reducing private information flow while preserving target accuracy and achieving high sparsity across four datasets.
Problem
Pruning is well established for efficiency, but its principled and measurable use to reduce privacy leakage remains underexplored.
Method
CutClean attaches auxiliary privacy heads to network blocks, measures leakage through their accuracy, and progressively selects and fine-tunes structured pruning levels.
Results
Across four datasets, CutClean reduced private information leakage while achieving high sparsity and preserving target-task accuracy, with consistent findings on vision transformers.
Takeaways & Limitations
The results support pruning as a privacy-preserving intervention that can combine leakage reduction with efficient model deployment.
Takeaways & Limitations
Privacy-head optimization can produce degenerate solutions that confidently misclassify private attributes while the backbone still reliably encodes them.
Abstract
from arXiv · showhide
Neural networks are increasingly deployed in high-stakes applications with growing privacy leakage concerns. We show that this privacy leakage can occur even in the absence of representation imbalances that lead to traditional dataset biases. This poses significant privacy risks when deploying models that process sensitive attributes. In this context, we propose CutClean, a privacy-aware pruning method that allows to reduce privacy information flow through the network, while increasing its sparsity. Our approach employs auxiliary linear privacy heads placed at each network's block to quantify information leakage, and further applies increasing levels of sparsity to remove the private attribute leakage, measured in terms of the accuracy of the privacy head attached to the last block. Experiments on synthetic and real-world datasets demonstrate that our approach effectively minimizes private information flow while achieving high sparsity rates and preserving classification target accuracy.
1 Introduction
Neural networks can encode and expose private information even without sensitive attributes or representation imbalances, creating risks in high-stakes deployments. CutClean addresses this problem through privacy-aware structured pruning that measures leakage at intermediate blocks while increasing sparsity and preserving target accuracy.
- Motivation: High-stakes neural-network deployments face privacy risks because internal representations can encode sensitive information that adversaries, auxiliary classifiers, or downstream tasks may extract.These risks remain even when models are trained without explicit access to sensitive attributes.
- Problem: Privacy leakage can persist without dataset imbalance or sensitive-target correlations, limiting mitigation strategies based on balancing, fairness constraints, or adversarial representation learning.Existing work has largely emphasized leakage arising as a byproduct of spurious correlations.
- Method: CutClean introduces privacy-aware structured pruning with auxiliary linear privacy heads attached to intermediate blocks to quantify private information at different network depths.The method directly measures leakage through privacy-head accuracy rather than relying on dataset-level assumptions.
- Method: CutClean progressively prunes the network to reduce private attribute leakage without adversarial minimax optimization, improving stability and post-deployment applicability.Leakage reduction is measured using privacy heads attached across network blocks.
- Results: CutClean achieves high structured sparsity while preserving target-task accuracy, indicating compatibility between privacy-leakage reduction and efficient model deployment.The framework positions pruning as a privacy-preserving intervention rather than solely a compression or robustness technique.
2 Related works
Prior work shows that neural-network representations can reveal sensitive attributes even when those attributes are excluded from training. CutClean addresses this leakage through diagnostic privacy heads and targeted pruning, distinguishing its goal from adversarial robustness.
- Privacy Leakage and Adversarial Attribute Inference: Neural-network representations can leak sensitive information despite protected attributes being excluded from training.Attribute-inference and auxiliary-classifier attacks recover attributes including gender, ethnicity, and health status from learned features with high accuracy.
- CutClean: CutClean uses pruning guided by explicit sensitive-attribute predictability measures rather than adversarial minimax optimization.Auxiliary linear privacy heads diagnose information flow at different network depths, while pruning removes components contributing to private-attribute leakage.
- CutClean: Unlike adversarial-robustness methods using pruned architectures, CutClean removes specific attributes that may be in-distribution during training rather than treating them as outliers.The distinction concerns the target of pruning: privacy attributes for CutClean versus adversarial attack robustness in prior work.
3 Method
CutClean prunes neural networks while preserving task performance and reducing private information flow. It attaches privacy heads to block outputs, minimizes a mutual-information proxy, and applies structured pruning under a privacy constraint.
- Privacy measurement: Auxiliary linear privacy heads attached to each block estimate the private information extractable from intermediate representations.Each privacy head is trained with private-label cross-entropy, while its performance measures private information from the corresponding block output.
- Privacy-aware objective: CutClean minimizes private information flow by constraining the last privacy head’s accuracy while balancing main-task performance and network sparsity.The last-block privacy-head accuracy serves as the proxy for private attribute flow, with the target being accuracy close to random guess.
- Structured pruning: The pruning scheme gradually increases global sparsity and removes entire channels using normalized L1-norm structured masks.Candidate sparsity levels are explored by grid search, and structured channel removal provides actual reductions in tensor dimensions and convolution operations.
- Privacy-aware objective: A mutual-information proxy avoids degenerate solutions caused by maximizing privacy-head error, forcing representations to discard private-attribute information.The regularization strength γ is tuned by selecting the value yielding the lowest validation accuracy of the last privacy head.
- Optimization procedure: Training alternates between optimizing privacy heads on private-label loss and updating the backbone and task head with the privacy-aware objective.The same batchwise alternating scheme is reused when fine-tuning pruned models.
4 Experiments
Experiments on synthetic and real-world datasets show that CutClean reduces privacy-head accuracy while preserving target performance and inducing sparsity. Ablations further examine architecture choice and fine-tuning, finding that fine-tuning improves target retention but can reduce selected sparsity.
- Datasets: Experiments span Corrupted-CIFAR10, Waterbirds, and CelebA, covering image corruptions, background, gender, ethnicity, “blonde,” and “heavy make-up” as private or target attributes.Custom dataset versions are balanced with respect to target and private attributes.
- Results on Corrupted-CIFAR10: Without MI-aware training, privacy-head accuracy remains around 64% on Corrupted-CIFAR10, whereas γ = 1000 provides the lowest privacy-head accuracy.The passage describes γ = 0 as the baseline and γ = 1000 as the best selection for reducing privacy leakage on this dataset.
- Results on Waterbirds: 91.47% privacy-head accuracy at γ = 0 on Waterbirds falls to 69.23% at γ = 1, which is selected for the remainder of the CutClean pipeline.Here, the private attribute is the background.
- Results on Waterbirds: 24.4% lower privacy-head accuracy and 20% sparsity are obtained on Waterbirds with a negligible 2.21% decrease in target accuracy.These results describe the pruned model reported after selecting γ = 1.
- Ablation studies: Fine-tuning pruned models retains more original target-classification performance, but selecting the highest-validation-accuracy model can result in significantly lower sparsity.The ablations also replicate the pipeline with a ViT-B16 model and evaluate candidate sparsity levels.
5 Conclusion
CutClean addresses privacy leakage and computational cost through privacy-aware neural-network pruning. Across four datasets, its admissible sparsity selection reduces private information leakage while choosing the best validation target accuracy.
- 5 Conclusion: CutClean combines privacy-aware pruning with the goal of reducing privacy leakage and improving neural-network efficiency.The method targets privacy leakage alongside the computational expense of deep neural networks.
- 5 Conclusion: Candidate sparsity levels are admissible when the last privacy-head accuracy falls below a threshold determined by the number of target classes.The procedure then selects the model with the best validation-set target accuracy.
- 5 Conclusion: Results on four datasets show that naive training exhibits significant private information leakage, which is reduced in the resulting pruned models.The conclusion reports this reduction as a central empirical outcome of the pruning procedure.