Source-linked AI summary
What Do Compressed Deep Neural Networks Forget?
Sara Hooker, Aaron Courville, Gregory Clark, Yann Dauphin, Andrea Frome
TL;DR
The paper examines whether top-line accuracy adequately captures what pruning and quantization change in deep neural networks. It audits impacts at class and exemplar levels across datasets, compression methods, and architectures, finding that compressed models diverge on a narrow subset of difficult, atypical, and underrepresented examples. The results motivate caution when deploying compressed models in sensitive domains and identify open questions about fairness and other domains.
Problem
Top-line accuracy may not capture how compression differentially affects classes and individual examples, despite widespread use of pruning and quantization.
Method
The paper audits compression impacts at class and exemplar levels through large-scale experiments across multiple datasets, compression techniques, and model architectures.
Results
PIEs are more challenging for humans and models, and compression disproportionately impairs prediction on the long-tail of less frequent instances.
Takeaways & Limitations
Compressed models should be used cautiously in sensitive domains where performance guarantees for particular dataset subsets matter.
Takeaways & Limitations
The study does not address implications for fairness or evaluate compression impacts in domains such as language and audio.
Abstract
from arXiv · showhide
Deep neural network pruning and quantization techniques have demonstrated it is possible to achieve high levels of compression with surprisingly little degradation to test set accuracy. However, this measure of performance conceals significant differences in how different classes and images are impacted by model compression techniques. We find that models with radically different numbers of weights have comparable top-line performance metrics but diverge considerably in behavior on a narrow subset of the dataset. This small subset of data points, which we term Pruning Identified Exemplars (PIEs) are systematically more impacted by the introduction of sparsity. Compression disproportionately impacts model performance on the underrepresented long-tail of the data distribution. PIEs over-index on atypical or noisy images that are far more challenging for both humans and algorithms to classify. Our work provides intuition into the role of capacity in deep neural networks and the trade-offs incurred by compression. An understanding of this disparate impact is critical given the widespread deployment of compressed models in the wild.
1 Introduction
The paper asks what compression removes beyond overall accuracy and introduces a framework for auditing class- and exemplar-level impacts. Across experiments, pruning disproportionately affects difficult, atypical, and less frequent examples despite small changes in top-line metrics.
- 1 Introduction: Compression can achieve high pruning and quantization levels with almost negligible loss to top-1 accuracy, while reducing memory, energy consumption, and inference latency.These benefits motivate compressed models in resource-constrained settings.
- 1 Introduction: Top-line test-set accuracy can hide how compression affects particular classes and examples.The paper asks whether compression costs are concentrated in a small subset rather than distributed uniformly.
- 1 Introduction: Pruning Identified Exemplars are images with high disagreement between pruned and non-pruned model predictions.Figure 1 shows ImageNet PIEs alongside non-PIE images from the same class, with true labels above each pair.
- 1 Introduction: PIEs are more difficult for models and humans, often involving mislabeled, low-quality, multi-object, or fine-grained images.The findings connect compression sensitivity with challenging visual examples.
- 1 Introduction: Compression impairs prediction on the long-tail of less frequent instances, motivating caution before deploying compressed networks in sensitive domains.The paper studies this pattern across CIFAR-10, CelebA, and ImageNet, using multiple compression techniques and architectures.
2 Methodology and Experiment Framework
The study evaluates how pruning and quantization affect class-level accuracy and image-level predictions beyond aggregate test-set metrics. It uses independent model populations across three datasets, compression variants, and architectures, with statistical tests to identify disparate impacts and PIEs.
- Compression representation: A sparse model sets fraction t of its weights to zero, with t = 0 denoting the non-compressed model.The framework also gives t = 0.9 the concrete interpretation that at most 10% of weights remain non-zero.
- Class-level analysis: Class-level impact is measured by comparing relative changes in class accuracy with the overall change in model accuracy.This controls for aggregate top-line accuracy differences when assessing whether compression affects particular classes disproportionately.
- Statistical testing: For each class, a two-tailed independent Welch’s t-test rejects the null hypothesis at p-value <= 0.05 when class-level changes differ significantly from overall accuracy changes.The method independently trains populations of K models and compares their class-accuracy distributions.
- Image-level analysis: A Pruning Identified Exemplar is an image whose modal prediction differs between the t-pruned model population and the non-pruned baseline population.PIE detection is unsupervised, can be performed at test time, and does not require the baseline prediction to match the true label.
- Experimental scope: The experiments cover CIFAR-10, ImageNet, and CelebA using wide ResNet, ResNet-50, and ResNet-18 models.Metrics are reported across 30 independent trainings for each dataset, model, and compression variant.
- Compression methods: Magnitude pruning varies end sparsity across t ∈ {0.3, 0.5, 0.7, 0.9}, while quantization is evaluated with float16, hybrid int8, and fixed-point int8 methods.Pruning is applied progressively during training; quantization is applied post-training.
3 Results
Across datasets and compression techniques, compression selectively harms a subset of classes and images rather than distributing its cost uniformly. These Pruning Identified Exemplars are harder to classify and often have atypical, ambiguous, or problematic visual attributes.
- 3.1 Disparate impact of compression: Compression disproportionately impacts a small, statistically distinguishable subset of classes, and this selective forgetting increases with sparsity.The number of significantly affected ImageNet classes rises from 170 at 50% sparsity to 372 at 70%.
- 3.1 Disparate impact of compression: Compression preserves similar overall accuracy partly by concentrating larger recall losses in a small subset of classes than the gains achieved elsewhere.The authors describe this pattern as cannibalizing performance on a small subset of classes.
- 3.1 Disparate impact of compression: Quantization introduces less disparate class-level harm than pruning: statistically significant recall differences affect 119 ImageNet classes with int8 weights versus 637 at 90% sparsity.These results suggest learned representations are more robust to reduced precision than to removing weights entirely.
- 3.1 Disparate impact of compression: The extent of disparate impact depends on task complexity and overparameterization; CIFAR-10 shows fewer significantly affected classes than ImageNet.At 30% and 50% sparsity, one CIFAR-10 class is affected, rising to two classes at 90%.
- 3.2 Pruning Identified Exemplars: PIEs are substantially harder for non-compressed models: top-1 accuracy falls from 76.75% to 39.81% on ImageNet, while non-PIE accuracy reaches 81.20%.The same PIE-versus-all contrast appears on CIFAR-10 and CelebA.
- 3.2 Pruning Identified Exemplars: Human annotations show PIEs over-index on incorrect labels, multiple objects, and fine-grained distinctions, including 59% multi-object images versus 39% of non-PIEs on ImageNet.These properties can make the classification task ambiguous or incorrectly specified.
4 Sensitivity of compressed models to distribution shift
The paper evaluates whether compression increases the brittleness of models under distribution shift using ImageNet-A and ImageNet-C. Pruning amplifies sensitivity to shifted inputs, with degradation varying substantially across corruption types and increasing at higher sparsity.
- 4 Sensitivity of compressed models to distribution shift: The motivation is practical: brittle behavior under distribution shifts can change model behavior in the wild and compromise human welfare.The paper frames relative robustness differences as relevant to the implications of compressed models for AI safety.
- 4 Sensitivity of compressed models to distribution shift: The analysis compares pruned and non-pruned models on ImageNet-A and ImageNet-C to measure relative sensitivity to distribution shift.ImageNet-A contains naturally adversarial images, while ImageNet-C applies algorithmically generated corruptions such as blur, noise, and fog.
- 4 Sensitivity of compressed models to distribution shift: Pruning greatly amplifies sensitivity to both ImageNet-A and ImageNet-C relative to non-pruned performance on the same inputs.For ImageNet-C, the comparison normalizes pruned-model top-1 accuracy by non-pruned accuracy for each corruption.
- 4 Sensitivity of compressed models to distribution shift: At t = 90, ImageNet-C relative top-1 degradation ranges from −40.11% for shot noise to −7.73% for brightness.Gaussian, shot, and impulse noise consistently produce among the highest relative degradation.
- 4 Sensitivity of compressed models to distribution shift: Sensitivity to small distribution shifts is amplified at higher sparsity, consistent with the view that excess capacity supports generalization on atypical or out-of-distribution inputs.The paper links this pattern to PIEs over-indexing on low-frequency attributes.
5 Related work
Prior compression research has largely evaluated trade-offs through overall accuracy, while this work examines class- and exemplar-level effects and robustness to natural distribution shifts.
- Compression trade-offs have predominantly been articulated through changes in overall accuracy at a given compression level.
- This work is presented as the first study to examine disaggregated model performance at both class and exemplar levels under compression.
- The paper measures compressed-model sensitivity to ImageNet-A and ImageNet-C distribution shifts.
- Its natural-shift analysis differs from adversarial-robustness studies that evaluate worst-case performance under targeted perturbations.
6 Discussion and Future Work
Compression methods are widely deployed under resource constraints, but their disparate effects can matter in sensitive applications. The paper connects capacity, atypical examples, human inspection, and unresolved fairness and domain-scope questions.
- Pruning and quantization are widely used in production because mobile and embedded deployments impose severe resource constraints.
- Top-line accuracy can hide compression effects on subsets for which sensitive applications may need guaranteed recall or performance.
- PIEs’ concentration on low-frequency attributes suggests that uncompressed networks use substantial capacity to represent these examples.
- Learning an appropriate mapping for a small subset may be better addressed in the data pipeline.
- The methodology can help humans understand compression trade-offs and surface challenging examples for judgment.
- The scope leaves open compression’s implications for fairness, language and audio domains, and explicit optimization against disparate impact.
A Pruning and quantization techniques considered
The paper considers magnitude pruning and post-training quantization as representative compression techniques, varying sparsity precisely and evaluating multiple weight representations.
- Magnitude pruning ranks weights by absolute value and removes those below a user-specified threshold.
- The evaluated pruning method follows Zhu and Gupta and enables precise control over final model sparsity.
- All networks are trained with 32-bit floating-point weights before post-training quantization without additional gradient updates.
- The quantization evaluation includes 16-bit floating-point and 8-bit integer weight representations, plus a third method using training examples.
B Pruning Protocol
The pruning protocol progressively targets specified sparsity levels during training, while quantization is applied after training without recalibration. CelebA additionally supports much higher pruning levels than the multiclass datasets.
- Target end-pruning levels are t ∈ {0.0, 0.1, 0.3, 0.5, 0.7, 0.9}.
- CelebA also reaches comparable final performance at t ∈ {0.95, 0.99}, unlike CIFAR-10 and ImageNet.
- The higher CelebA pruning tolerance may relate to its binary hair-color classification task rather than the 10- or 1000-class settings.
- Quantization is applied post-training, with weights not recalibrated afterward; model-population accuracy distributions are shown for three datasets.
C Human study
The human study compares PIE and non-PIE images to identify which image attributes are overrepresented among examples affected by pruning. PIEs over-index on noisy, atypical, and challenging examples, while the broader experiments track pruning across datasets and classes.
- Human study: The human study used 85 volunteers to inspect a balanced random sample of PIE and non-PIE images without revealing each image’s category.Participants evaluated image-label correctness, object count, semantic similarity, typicality, corruption, and abstract representation.
- Human study: PIEs heavily over-index on noisy examples with incorrect labels, multiple objects, or corrupted image information.The study also classified common image-quality corruptions such as blur, fog, noise, and compression artifacts.
- Human study: PIEs also over-index on atypical or challenging images involving fine-grained classification and abstract representations.These attributes were measured through human judgments of semantic closeness, typicality, and abstraction.
- Cross-dataset evaluation: The CIFAR-10 and ImageNet experiments report top-1 accuracy across pruning levels and count the classes significantly impacted by pruning.CIFAR-10 top-5 accuracy was 99.8% at all pruning levels; CelebA is separately evaluated as a two-class task.
D Benchmarks to evaluate robustness
The robustness benchmarks evaluate how pruning affects performance on corrupted and natural-adversarial images. Increasing pruning amplifies sensitivity to distribution shift, with stronger degradation on ImageNet-C and ImageNet-A.
- ImageNet-A: ImageNet-A contains 7,500 natural adversarial images designed to produce drastically low test accuracy.The benchmark reports absolute and normalized top-1 and top-5 performance at different sparsity levels.
- ImageNet-A: As pruning increases, ImageNet-A top-1 and top-5 accuracy further erode, indicating greater brittleness to adversarial examples.The sensitivity pattern mirrors the degradation observed on ImageNet-C and among PIEs.
- Evaluation procedure: Robustness results are averaged over five independently trained models for each benchmark and pruning level.This averaging procedure reduces dependence on a single random initialization.
- ImageNet-C: ImageNet-C evaluates pruned and non-pruned models on algorithmically corrupted ImageNet images across five corruption severity levels.The benchmark includes corruptions such as blur and noise, and reports performance across corruption types.
- ImageNet-C: Pruning substantially degrades mean top-1 accuracy on ImageNet-C relative to non-pruned models, especially at high pruning levels.Sensitivity varies across corruption types, with Gaussian, shot, and impulse noise producing consistently greater degradation.