Source-linked AI summary
A Closer Look at Memorization in Deep Networks
Devansh Arpit, Stanisław Jastrzębski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S. Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, Simon Lacoste-Julien
TL;DR
The paper examines why DNNs generalize despite being able to memorize noise, focusing on whether gradient-based training behaves differently on real and random data. Through controlled experiments and regularization comparisons, it finds that DNNs learn simple patterns first and that data-dependent memorization helps explain generalization beyond dataset-independent effective capacity.
Problem
Over-parameterized DNNs can memorize random data yet often generalize well, leaving unclear whether effective capacity alone explains their generalization.
Method
The paper contrasts DNN learning dynamics on real, random-input, and random-label datasets, and evaluates how regularization affects memorization and real-data learning.
Results
DNNs learn simple and general patterns before fitting noise, while dropout can hinder memorization without reducing validation accuracy on clean data.
Takeaways & Limitations
Memorization and generalization depend on architecture, optimization procedure, and the training data itself, so dataset-independent effective capacity is unlikely to fully explain DNN generalization.
Takeaways & Limitations
The interpretation is constrained by training-point behavior under log loss, which pushes correctly classified points away from the decision boundary and affects critical-sample counts.
Abstract
from arXiv · showhide
We examine the role of memorization in deep learning, drawing connections to capacity, generalization, and adversarial robustness. While deep networks are capable of memorizing noise data, our results suggest that they tend to prioritize learning simple patterns first. In our experiments, we expose qualitative differences in gradient-based optimization of deep neural networks (DNNs) on noise vs. real data. We also demonstrate that for appropriately tuned explicit regularization (e.g., dropout) we can degrade DNN training performance on noise datasets without compromising generalization on real data. Our analysis suggests that the notions of effective capacity which are dataset independent are unlikely to explain the generalization performance of deep networks when trained with gradient based methods because training data itself plays an important role in determining the degree of memorization.
1. Introduction
The paper asks why over-parameterized DNNs can generalize despite their capacity to memorize, and investigates how optimization and data content shape memorization. Its experiments distinguish learning shared patterns from brute-force memorization and assess whether regularization can selectively hinder memorization.
- 1. Introduction: Over-parameterized DNNs often generalize well despite traditional theory associating sufficient capacity with memorization and poor generalization.The resulting tension is not explained by representational capacity alone.
- 1. Introduction: Representational capacity describes expressible hypotheses, whereas effective capacity concerns hypotheses reachable by a specified model and training procedure.The paper formalizes effective capacity as EC(A) = {h | ∃D such that h ∈ A(D)}, with A(D) denoting hypotheses reachable on dataset D.
- 1. Introduction: Gradient-based training does not sufficiently prevent memorization, because DNNs can fit pure noise without substantially longer training.This challenges dataset-independent capacity explanations for generalization.
- 1. Introduction: The paper operationalizes memorization through DNN behavior on random data and contrasts optimization on real versus noise datasets.This approach tests whether real-data fitting relies on shared patterns rather than content-independent memorization.
- Main Contributions: DNNs learn simple patterns before memorizing, and regularization can hinder memorization while preserving learning from real data.These findings support content-aware optimization rather than simple memorization of real datasets.
2. Experiment Details
The experiments use MNIST and CIFAR10 with MLP and CNN models, replacing inputs or labels with controlled random data to compare learning on real and noisy datasets.
- 2. Experiment Details: Experiments use MNIST with two-layer ReLU MLPs and CIFAR10 with small AlexNet-style CNNs.The MLPs generally have 4096 hidden units per layer; CNN training uses momentum 0.9 and a scheduled learning rate.
- 2. Experiment Details: The study replaces portions of labels with random labels or inputs with Gaussian noise matched to the real data’s mean and variance.randX denotes noisy inputs, while randY denotes noisy labels; unless specified, the noisy portion is 100%.
3. Qualitative Differences of DNNs Trained on Random vs. Real Data
DNNs optimize differently on real and random data: real datasets produce evidence of shared-pattern learning, while random inputs require broadly distributed memorization. Capacity and training-time experiments further show that noise changes validation and convergence behavior.
- Optimization on Real vs. Random Data: DNNs fit random data but do not use the same brute-force memorization strategy on real datasets.Experiments compare real data with random inputs or labels to expose qualitative optimization differences.
- Easy Examples as Evidence of Patterns in Real Data: Real datasets contain consistently easy and hard examples after one epoch, whereas randX difficulty differences are well modeled as random Binomial noise.The results support the conjecture that simple patterns explain easier real examples early in training.
- Loss-Sensitivity in Real vs. Random Data: For real data, only a subset of examples has high loss-sensitivity, while random data produces high sensitivity for virtually all examples.The proxy averages the loss-gradient norm with respect to previous training examples across unrolled SGD updates.
- Loss-Sensitivity in Real vs. Random Data: Class-specific loss-sensitivity is highest within the same class but more spread across classes for real data, suggesting richer cross-category patterns.Figure 4 compares per-class gradient sensitivities for real and random data on a log scale.
- Effects of Capacity and Dataset Size on Validation Performances: Higher capacity is needed to achieve optimal validation performance when noise examples are present, although random labels on CIFAR10 show no capacity relationship.The capacity trend is reported for noisy MNIST and noise inputs on CIFAR10, but not for random labels on CIFAR10.
- Effects of Capacity and Dataset Size on Training Time: Training-time effects are stronger for noise: reducing capacity or increasing dataset size slows convergence more severely, while real data shows stronger diminishing returns from added capacity.These results suggest the networks extract patterns from real data rather than relying only on memorization.
4. DNNs Learn Patterns First
DNNs learn progressively more complex hypotheses, with real-data patterns learned before noise is memorized. Critical sample ratios are higher for noise-trained models and increase as training proceeds.
- Critical Sample Ratio: A higher critical sample ratio indicates a more complex hypothesis because more data points lie near decision boundaries.CSR is the fraction of dataset points for which a nearby adversarial example can be found.
- Critical Sample Ratio: LASS searches within a radius-r box for nearby adversarial examples and uses noise to escape zero-gradient points that may have large second derivatives.This extends FGSM by exploring the search box more thoroughly.
- Critical Sample Ratio: Critical sample ratios increase during training and then stabilize, indicating gradually more complex learned hypotheses across real, randX, and randY datasets.The study measures critical samples in validation sets throughout training.
- Results: Noise-trained models have more critical samples than real-data models, suggesting that their learned decision surfaces are more complex.The comparison covers random inputs and random labels.
- Results: With 20% to 80% of training data replaced by input or label noise, noisier datasets show higher CSR and lower final and maximum validation accuracy.The pattern is reported for both MNIST and CIFAR-10.
- Results: On random-label datasets, validation accuracy peaks before training accuracy becomes high, while CSR rises as the model fits the noise.This supports the conclusion that real examples are easier to fit than noise.
5. Effect of Regularization on Learning
Explicit regularization can slow memorization of random labels while preserving generalization on clean data. The experiments compare several regularizers using training accuracy on randY and validation accuracy on CIFAR-10.
- Findings: Regularization can degrade training performance on random-label data while maintaining generalization performance on real data.The authors report this as evidence that explicit regularization can limit noise memorization without significantly affecting real-data learning.
- Evaluation: The comparison includes dropout, input dropout, Gaussian noise, weight decay, and dropout with adversarial training.Regularization parameters are varied separately for each dataset, technique, and model combination.
- Evaluation: Figure 10 plots final randY training accuracy against the best CIFAR-10 validation accuracy across regularization parameters.Flat curves indicate reduced random-label memorization with unchanged clean-validation accuracy.
6. Related Work
Prior work established that DNNs can fit random data, while this paper examines how optimization, regularization, and dataset properties shape memorization and learning dynamics.
- DNNs can fit random data, challenging explanations of generalization based solely on traditional statistical learning theory.This work builds on Zhang et al.'s noise-fitting experiments while challenging their interpretation of memorization and regularization.
- Figure 11 shows that regularizers have different effects on training curves for random-label and real data, indicating different propensities to slow memorization.
- Regularizers, especially dropout, can control the speed at which DNNs memorize random data.The paper connects this effect to dropout's known role in preventing catastrophic forgetting.
- Earlier work suggested that SGD and early stopping preferentially learn simpler hypotheses before more complex ones.
- This paper studies how the fraction of noise samples relates to capacity, training time, and dataset size.
- The analysis compares training complexity across datasets and regularizers using critical sample ratios rather than loss-minimum sharpness.Critical samples are real data points with nearby adversarial examples, following the paper's adopted definition of adversarial examples.
7. Conclusion
The empirical results indicate that DNNs trained with SGD variants prioritize patterns over brute-force memorization when fitting real data, despite being able to fit noise. The findings leave the source of generalizable solutions unclear and suggest that distributed and hierarchical representations may play an important role.
- DNN optimization differs qualitatively between noise and real data.
- DNNs trained with SGD variants first use patterns, rather than brute-force memorization, to fit real data.
- Although DNNs can fit noise, the experiments do not explain why they find generalizable solutions on real data.
- Distributed and hierarchical representations are proposed as deep-learning priors that may help explain this behavior.