Source-linked AI summary
Predicting Privacy Leakage from Weight Spectral Density
Richard J. Preen, Jim Smith
TL;DR
State-of-the-art membership inference audits can require computationally expensive shadow models, so the paper tests whether inexpensive WeightWatcher spectral metrics can proxy privacy vulnerability. Across image and tabular tasks, stable rank tracks overall MIA success and Log α-Norm tracks low-false-positive vulnerability more consistently than the generalisation gap, supporting scalable spectral auditing.
Problem
State-of-the-art membership inference attacks require expensive shadow models, while the generalisation gap is an inexpensive but unreliable privacy-risk proxy.
Method
The paper evaluates WeightWatcher spectral metrics derived from heavy-tailed self-regularisation on image and tabular classification tasks and compares them with the generalisation gap.
Results
Stable rank is strongly associated with LiRA AUC, while Log α-Norm has a consistent negative association with LiRA TPR@0.001; these relationships exceed those observed for the generalisation gap.
Takeaways & Limitations
Spectral metrics provide a promising computationally efficient signal for scalable, data-free privacy-risk assessment and complement generalisation-based metrics.
Takeaways & Limitations
The study covers only CIFAR-10 and Volkert, uses MLPs and one MIA method, and does not establish causality between spectral properties and privacy leakage.
Abstract
from arXiv · showhide
Membership inference attacks (MIAs) are widely used to audit the privacy disclosure risk of machine learning models, however current state-of-the-art attacks require training computationally expensive shadow models, making large-scale privacy evaluation impractical. In this work, we investigate whether inexpensive spectral metrics derived from the heavy-tailed self-regularisation framework can serve as proxies for MIA vulnerability. We evaluate several WeightWatcher spectral metrics on image and tabular classification tasks and compare their relationship with MIA privacy leakage against conventional measures of generalisation. Across datasets, stable rank exhibits a strong positive correlation with overall MIA success, while Log alpha-Norm shows a consistent negative correlation with MIA vulnerability at the low false-positive regime. These associations are observed to be stronger than those obtained using the generalisation gap. The results indicate that neural network spectra may contain information about privacy leakage that is not fully captured by conventional measures of overfitting, motivating spectral analysis as a promising direction for scalable privacy auditing.
1 Introduction
Membership inference attacks provide standard privacy audits but state-of-the-art versions require expensive shadow models, motivating cheap, data-free spectral proxies. This paper investigates WeightWatcher metrics and finds associations with MIA vulnerability that complement and can outperform the generalisation gap.
- Current strong MIAs such as LiRA and RMIA typically train reference models, making broad screening computationally expensive, especially when many candidate models require auditing.
- The study provides the first empirical investigation of relationships between neural-network weight spectral metrics and MIA vulnerability.
- WW stable rank is strongly associated with overall MIA success, with a stronger association than the generalisation gap.
- WW Log α-Norm is consistently associated with MIA success in the low false-positive-rate regime, where the generalisation gap is weak and inconsistent.
- Combining spectral metrics with the generalisation gap improves prediction of privacy vulnerability over either signal alone.
- Spectral metrics can identify high-risk models as a binary classification task, indicating potential utility for scalable, data-free privacy auditing.
2 Background
WeightWatcher analyzes trained weight spectra without data access, while existing MIA approaches trade attack strength against computational cost. The generalisation gap is an inexpensive but unreliable privacy proxy, motivating spectral metrics as complementary alternatives.
- 2.1 Heavy-Tailed Self-Regularisation and WeightWatcher: WW computes spectral summary statistics from trained weights alone, offering data-free proxies for model generalisation and potential privacy leakage.
- 2.1 Heavy-Tailed Self-Regularisation and WeightWatcher: WW fits layer ESD tails to power laws and reports α-based norm metrics and stable rank to characterize spectral structure and effective dimensionality.
- 2.2 Background: Strong MIAs such as LiRA use multiple shadow models to estimate membership vulnerability, whereas alternatives reduce retraining through auxiliary data, retained training trajectories, or other signals.
- 2.2.3 Cheap-to-compute privacy proxy metrics: The generalisation gap is a cheap training-free privacy proxy, but large expressive models can leak membership information without that leakage appearing in aggregate held-out performance.
- 2.2.3 Cheap-to-compute privacy proxy metrics: Table 1 distinguishes WW metrics computed directly from weights without data from target-model and MIA metrics requiring data and/or shadow models.
- 2.2 Background: Existing reference-model attacks offer reliable leakage estimates at high computational cost, while reference-free methods often require auxiliary data, training trajectories, or coarse aggregate estimates.
3 Methodology
The study evaluates whether WeightWatcher spectral metrics can provide inexpensive, data-free indicators of privacy leakage, using models varied across architectures and training settings. It compares spectral measures with generalisation and LiRA-based membership-inference metrics on image and tabular classification tasks.
- Evaluation metrics: The evaluation compares generalisation, WeightWatcher spectral complexity, and privacy leakage measured with online LiRA using 64 shadow models.LiRA reports AUC and TPR@0.001, the true-positive rate at a fixed false-positive rate of 0.001.
- Spectral metrics: WeightWatcher analyses each layer’s correlation matrix X = W^T W without training or test data, fits a truncated power law to its spectral-density tail, and aggregates layer-level statistics.The evaluated metrics are α, Log α-Norm, Log spectral norm, and stable rank.
- Spectral metrics: Stable rank measures effective weight-matrix dimensionality; higher values indicate more random-like capacity and, under HT-SR theory, a greater tendency toward memorisation.It is presented as a robust, noise-tolerant alternative to ordinary matrix rank.
- Datasets: The study uses CIFAR-10 image classification and the 10-class Volkert tabular dataset, with standard dataset-specific train/test splits and normalised Volkert features.CIFAR-10 uses 50,000/10,000 examples, while Volkert uses a stratified 80-20% split.
- Experimental setup: The experiments train 44 target models spanning 11 architectures, two learning rates, and two weight-decay values, evaluated after 100 epochs.Models are feedforward ReLU MLPs without dropout or normalisation, using SGD with momentum.
4 Results
Across CIFAR-10 and Volkert, stable rank is most strongly associated with overall LiRA attack success, while Log α-Norm is more informative for low-false-positive leakage. Spectral and generalisation measures provide complementary signals, but low-FPR prediction remains harder and less consistent across datasets.
- Spectral metrics and privacy leakage: Stable rank correlates with LiRA AUC at ρ = 0.87 on CIFAR-10 and ρ = 0.60 on Volkert, exceeding the generalisation-gap correlations of ρ = 0.67 and ρ = 0.54.These associations are reported as statistically significant at p ≤ 0.01.
- Spectral metrics and privacy leakage: Log α-Norm correlates negatively with LiRA TPR@0.001 at ρ = −0.55 on CIFAR-10 and ρ = −0.40 on Volkert, while the generalisation gap is weaker and inconsistent.TPR@0.001 is shown on a log scale in the dataset figures.
- Combined predictors: Combining stable rank with the generalisation gap explains R2 = 0.71 on CIFAR-10 and R2 = 0.57 on Volkert for LiRA AUC, outperforming either predictor alone.The combined model is supported by significant contributions from both predictors on Volkert, although the larger standardised effect reverses across datasets.
- Combined predictors: Log stable rank raises combined AUC prediction to R2 = 0.82 on CIFAR-10 and R2 = 0.62 on Volkert, contributing at least as much predictive weight as the generalisation gap.The transformed stable-rank model also resolves the coefficient-ordering reversal reported for raw stable rank.
- Low-FPR prediction: For low-FPR leakage, combined models explain only R2 = 0.19 on CIFAR-10 and R2 = 0.16 on Volkert, indicating substantially harder and less consistent prediction than LiRA AUC.On Volkert, the generalisation gap is marginally significant while Log α-Norm falls just short of significance.
- High-risk classification: Combining Log α-Norm with the generalisation gap improves cross-dataset high-risk classification, converging at ROC-AUC 0.81 versus 0.82 at the 20× threshold.Standalone metrics diverge more strongly across domains, especially for Volkert.
5 Conclusions and Limitations
Across image and tabular datasets, WeightWatcher metrics show useful associations with MIA privacy leakage, often stronger than the generalisation gap, while combining signals improves prediction. These findings support scalable, data-free privacy auditing but remain limited by dataset, attack, and modeling scope and do not establish causality.
- Stable rank is strongly associated with LiRA AUC, while Log α-Norm is consistently negatively associated with LiRA TPR@0.001 across image and tabular datasets.
- Spectral relationships with privacy leakage are stronger than those observed for the conventional ϵgap.
- Combining ϵgap with (log) stable rank substantially improves LiRA AUC prediction on both datasets, with both coefficients remaining jointly significant.
- α and Log α-Norm reliably separate high-risk from low-risk models on both datasets, whereas ϵgap and stable rank do not generalise consistently across domains.
- Spectral metrics provide a promising computationally efficient signal for privacy assessment from trained weights, supporting potential scalable auditing and privacy-oriented model selection.
- The evidence does not establish causality and is limited to two datasets, one attack method, small regressions, and no cross-dataset transfer or held-out validation.