PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Learning Through the Lens of Example Difficulty

Robert J. N. Baldock, Hartmut Maennel, Behnam Neyshabur

arXiv:2106.09647v2cs.LGstat.ML

TL;DR

The paper asks how to characterize the computational difficulty of individual examples beyond one-dimensional or data-aggregated measures. It introduces prediction depth from hidden embeddings, finds broad relationships with accuracy, consistency, margins, uncertainty, and learning speed, and uses difficulty categories to improve predictions while unifying several deep-learning observations.

  • Problem

    Existing example-difficulty notions do not capture processing inside a converged model and cannot distinguish examples that are difficult for different reasons.

  • Method

    The paper defines prediction depth from hidden embeddings and combines training- and validation-split depths to identify interpretable difficulty types and guide hidden-layer prediction.

  • Results

    Prediction depth relates to prediction consistency, accuracy, margins, and learning speed, while difficulty-specific hidden-layer predictions can produce dramatic accuracy improvements.

  • Takeaways & Limitations

    The results unify early-layer generalization, later-layer memorization, layerwise convergence, and the earlier learning of easy data and simpler functions.

  • Takeaways & Limitations

    The study tests image-classification settings and does not establish that its findings extend to larger models, larger datasets, or other tasks.

Abstract

from arXiv · show

Existing work on understanding deep learning often employs measures that compress all data-dependent information into a few numbers. In this work, we adopt a perspective based on the role of individual examples. We introduce a measure of the computational difficulty of making a prediction for a given input: the (effective) prediction depth. Our extensive investigation reveals surprising yet simple relationships between the prediction depth of a given input and the model's uncertainty, confidence, accuracy and speed of learning for that data point. We further categorize difficult examples into three interpretable groups, demonstrate how these groups are processed differently inside deep models and showcase how this understanding allows us to improve prediction accuracy. Insights from our study lead to a coherent view of a number of separately reported phenomena in the literature: early layers generalize while later layers memorize; early layers converge faster and networks learn easy data and simple functions first.

1 Introduction

The paper addresses limits of data-aggregated measures of deep learning by introducing prediction depth, an example-level measure based on hidden representations. It relates example difficulty to model behavior and distinguishes interpretable difficulty types while connecting these findings to broader phenomena in deep learning.

  • Motivation: Existing difficulty measures do not capture how a converged model processes individual examples or distinguish examples that are difficult for different reasons.The paper motivates an example-level perspective because prior notions omit internal processing and largely provide a one-dimensional difficulty view.
  • Difficulty types: Three interpretable difficulty types and four extreme combinations of training- and validation-split difficulty reveal different ways examples can be difficult.A middle-layer hidden k-NN algorithm can produce dramatic accuracy improvements for inputs strongly exhibiting one specific difficulty type.
  • Prediction depth: Prediction depth measures the number of hidden layers after which a network’s final prediction is effectively determined.It is computed from hidden embeddings and is intended as a measure of computational example difficulty.
  • Empirical findings: Prediction depth is larger for visually difficult examples and remains consistent across architectures and random seeds.Figure 1 illustrates early predictions for visually typical examples and late predictions for mislabeled or confusing examples.
  • Empirical findings: Predictions with smaller prediction depths are more accurate on average, while prediction depth appears to provide a linear lower bound on prediction consistency.The paper also relates prediction depth to uncertainty, confidence, margins, and learning speed.
  • Broader implications: The findings provide a coherent account of early-layer generalization, later-layer memorization, layerwise convergence, and the earlier learning of easy examples and simpler functions.The experimental study spans ResNet18, VGG16, and MLP models trained on CIFAR10, CIFAR100, Fashion MNIST, and SVHN.

2 Prediction Depth: a Computational View of Example Difficulty

Prediction depth provides a computational view of example difficulty by identifying when a network’s final prediction is effectively determined, using layerwise embedding probes. It is defined for both training and validation examples and passes sanity checks linking larger depths to more difficult examples and showing consistency across architectures and random seeds.

  • Definition: Prediction depth measures the number of hidden layers after which a network’s final prediction is effectively determined.It provides a computational perspective in which easy examples are predicted after fewer layers, while difficult examples use more layers.
  • Definition: The method constructs k-NN probes from training-set embeddings after selected network layers, including the input and final softmax, using k = 30.A prediction is assigned to layer l when the preceding probe disagrees with the final classification but all probes from layer l onward agree with it.
  • Definition: Prediction depth can be calculated for both training and validation points, distinguishing difficulty of predicting a given class from difficulty of predicting an unseen input.The training notion concerns the given class, whereas the validation notion concerns an input unseen in advance.
  • Meaningful and robust difficulty: Prediction depth passes both sanity checks: visually confusing or mislabeled examples should have larger depths, while intuitively simpler tasks should have lower average depths.Figure 1 is reported to show both relationships.
  • Meaningful and robust difficulty: Prediction depth is highly consistent across random seeds and architectures, with stronger correlation between the convolutional ResNet18 and VGG16 than between VGG16 and MLP.The difference is attributed to the more similar inductive biases of the two convolutional architectures; perfect agreement is not expected.

3 Deep Learning Phenomena Through the Lens of Prediction Depth

Prediction depth connects computational example difficulty to prediction consistency, accuracy, learning speed, and the margins and layers involved in processing.

  • Accuracy and consistency: The consistency score measures how often an example is classified correctly when omitted from training, while consensus-consistency measures agreement with an ensemble’s consensus class.The consistency score averages correctness across models trained on random subsets excluding the example; consensus-consistency is the fraction predicting the ensemble consensus class.
  • Accuracy and consistency: Prediction depth forms a linear lower bound on consensus-consistency, with lower-depth predictions more likely to be consistent and correct.High-depth predictions are more sensitive to training splits and random seeds, while low-depth predictions align more often with the consensus class.
  • Learning order: Data points with smaller prediction depths are generally learned earlier, and training learning curves share important features with inference learning curves from hidden-layer k-NN probes.With corrupted labels, both curves show initial processing according to original classes followed by learning of the randomized labels.
  • Margins and hidden-layer processing: Reducing the output margin produces models whose data clustering and predictions are determined only in later hidden layers, yielding high prediction depths.The intervention used a loss and optimizer that did not encourage high output margins; the authors report the intervention and outcome without identifying all contributing factors.

4 Beyond a One-Dimensional Picture of Example Difficulty

The paper replaces one-dimensional example difficulty with two prediction-depth axes and four interpretable combinations, revealing distinct internal processing patterns. These distinctions also support a targeted accuracy intervention for examples ambiguous without their labels.

  • Two axes of example difficulty: Prediction depth differs between validation and training splits, defining difficulty of prediction and difficulty of finding same-class commonalities.The axes are PDVal. and PDTrain, each ranging from clear to ambiguous.
  • Four difficulty types: Easy examples have low PDVal. and low PDTrain, are often visually typical, and nearly always receive their ground-truth label.
  • Four difficulty types: Examples that look like a different class have low PDVal. but high PDTrain, yielding clear yet nearly always incorrect validation predictions.Mislabeled and visually confusing examples are included in this category.
  • Four difficulty types: Examples ambiguous unless the label is given have high PDVal. and low PDTrain, making them difficult to connect to their predicted class in validation.They may visually resemble both their own and another class and are likely to be misclassified.
  • Four difficulty types: Ambiguous examples have high PDVal. and high PDTrain, with low early-layer confidence and predictions that can depend strongly on training randomness.Later layers recognize them with intermediate confidence and accuracy as the consensus class.
  • Improving prediction accuracy: For the 100 examples most ambiguous without their labels, layer-4 k-NN increased accuracy from 25% to 98% relative to the full model.The comparison used CIFAR10 with ResNet18 and contrasted an intermediate-layer probe with the full model.

5 Discussion

The discussion connects prediction depth to previously separate findings about consistency, learning order, margins, and layer-wise processing. It also frames the measure as relevant to curriculum learning, distribution shift, and fairness while stating important scope limitations.

  • Connecting known phenomena: The paper presents prediction depth as a coherent explanation linking early-layer generalization, later-layer memorization, layer-wise convergence, and learning easy data and simple functions first.
  • Connecting known phenomena: Early-layer predictions are more consistent and likely correct, whereas later-layer predictions are less consistent and have lower expected accuracy.
  • Connecting known phenomena: Examples learned early in training typically have smaller prediction depths than examples learned later.
  • Connecting known phenomena: Networks exhibit wider input and output margins on average near data with smaller prediction depths.The paper treats these margins as common measures of local simplicity.
  • Relevance to machine learning topics: The framework is presented as relevant to curriculum learning, robustness to distribution shifts affecting subgroup frequencies, and heteroscedastic uncertainty.The paper expands these connections in an appendix.
  • Limitations: The reported findings have not been tested in larger models, larger datasets, or tasks beyond image classification, and practical applications are otherwise largely unexplored.The authors also caution that simply reducing prediction depth is not sufficient for good generalization.

A Detailed Description of Experiments, Architectures and Hyperparameter Optimization

Experiments span four datasets and three architectures, using repeated random validation splits and additional full-training runs. The architectures include modified ResNet18 and VGG16 models plus a seven-hidden-layer MLP.

  • Experimental setup: For every dataset–architecture combination, the authors trained 250 models with random 10% validation splits and 25 additional models on the full training set.
  • Experimental setup: The study used CIFAR10, CIFAR100, Fashion MNIST, and SVHN with ResNet18, VGG16, and MLP architectures.
  • Architectures: ResNet18 replaced Batch Norm with Group Norm and used Weight Standardization.
  • Architectures: VGG16 removed its final three dense layers and used neither batch normalization nor dropout.The modification was described as standard for datasets smaller than ImageNet.
  • Architectures: The MLP used seven ReLU hidden layers of width 2048, selected after comparing seven- and fifteen-layer networks on CIFAR10 with 40% fixed random label noise.

A.2.4 Data augmentation

The experimental appendices specify training, probe placement, consensus estimation, and a margin intervention. They report probe robustness across k and runs, while showing that the combined loss-and-optimizer intervention produces the smallest margin and latest-layer clustering.

  • Data augmentation: Data augmentation was not applied because different schemes could affect examples differently; its use was left for subsequent studies.
  • Hyperparameter optimization: The training sweep varied learning rate, momentum, and weight decay, then selected seven accurate and stable curves for extended training.At least two optimization rounds adapted the learning-rate schedule, using batch size 256.
  • Probe robustness: For VGG16 on CIFAR10, k-NN probe accuracies were insensitive to k at k = 30, and independent-run results showed tight spreads.
  • Probe construction: Prediction-depth probes were placed after architecture-specific operations, including dense layers, convolutions, residual-block sums, normalization, and softmax.
  • Prediction-depth assignment: The full model and post-softmax k-NN predictions nearly perfectly agreed; prediction depth was unassigned in rare disagreements.Such disagreements occurred fewer than 1 in 10^4 cases.
  • Consensus-consistency estimation: The same ensemble estimated both consensus class and consensus-consistency, creating a biased estimator of the consistency quantity.An unbiased estimator would require an additional model ensemble at greater cost.

B Further Related Work

The paper situates prediction depth among example-level difficulty measures and related analyses of how individual data points are learned and represented. It distinguishes its computationally derived difficulty categories from human-defined intuitive categories and relates prediction depth to prior methods using hidden-layer probes.

  • Existing approaches: Prior work studies example difficulty through gradient interference, partially randomized labels, ensemble behavior, privacy impact, pruning, forgetting, and validation accuracy.These approaches use multiple observables and training outcomes to characterize individual data points.
  • Connections to prior findings: The order in which data points are learned is shared across architectures and random seeds, paralleling prediction depth consistency.The paper connects this prior observation to its own correlation between prediction depth and learning order.
  • Forms of difficulty: Hooker et al. define difficulty using four human-assigned categories: incorrect labels, multiple-object images, corrupted images, and fine-grained classification.The paper contrasts these intuitive categories with its own model-derived forms of computational difficulty.
  • Hidden-layer methods: Deep k-Nearest Neighbors classifies examples by aggregating nearest-neighbor class labels from probes applied across the network’s hidden spaces.This method provides related context for the paper’s use of hidden representations to analyze predictions.

C.1 Consistency of prediction depth between architectures

Across datasets and architectures, the supplementary experiments examine whether prediction depth is consistent and how it relates to prediction entropy, consistency, and learning order. They reproduce the main relationships using ensembles and repeated random train-validation splits.

  • Consistency across architectures: 250 models per dataset-architecture combination provide histograms comparing mean prediction depths between architectures on training and validation splits.The models use random 90:10% splits, and the corresponding figures report Spearman correlations beneath each plot.
  • Prediction consistency: Easier classification tasks are solved after fewer layers, as reflected by the gradients of the reported linear bounds.Figures C.17 and C.18 reproduce the main consistency and prediction-depth relationships across datasets and architectures.
  • Prediction entropy: Prediction entropy is computed from the normalized histogram of one-hot ensemble predictions for an input excluded from the models’ training subsets.For N classes, p_i(x) denotes the fraction of ensemble models predicting class i.
  • Prediction entropy: Mean prediction depth defines a linear upper bound on prediction entropy, paralleling the linear lower bound on consensus consistency.The relationship is shown for validation examples across datasets and architectures.
  • Learning order: Supplementary figures compare mean prediction depth with learning iteration and reproduce these relationships across architectures, datasets, and training or validation splits.The repeated evaluations are intended to broaden the picture of how prediction depth relates to when examples are learned.

C.4 Consistency of margin results

The supplementary experiments reproduce the margin-related behavior of the main analysis across all datasets and architectures in both training and test splits.

  • Margin consistency: Figures C.28 to C.31 reproduce the left and middle panels of Figure 6 for every dataset and architecture in training and test splits.The passage identifies the scope of replication but does not state an additional quantitative outcome.

C.5 Consistent two-dimensional relationship between prediction depths in the training and validation splits

The supplementary analysis tests whether the two-dimensional relationship between prediction depths in training and validation remains consistent across datasets and architectures. It also reproduces the behavior illustrated in the main figures for CIFAR10 and CIFAR100.

  • Training-validation relationship: Prediction-depth histograms compare training and validation splits for all data points, whose depths can differ depending on the consensus class.Across each dataset and architecture, 250 models use random 90:10% validation-to-training splits and record depth for each appearance.
  • Training-validation relationship: Figures C.36 to C.47 reproduce behavior analogous to Figure 8 across all datasets and architectures.Figure 8 contains the detailed description of the behavior being replicated.
  • Cross-architecture replication: Figure C.17 demonstrates consistency of the Figure 3 and Figure 4 (left) behavior for both CIFAR10 and CIFAR100 across architectures.The passage identifies the datasets and cross-architecture scope of the reproduction.

D Pertinence of example difficulty to topics in machine learning

The paper connects its richer account of example difficulty to distribution shift, fairness, uncertainty modeling, and curriculum learning, while presenting supporting consistency analyses across datasets and architectures.

  • Distribution Shift and Robustness: Richer example-difficulty measures may deepen understanding of distribution shift and aid development of more robust algorithms.
  • Curriculum Learning: Curriculum learning presents easier examples earlier, and richer difficulty distinctions could support more effective curricula.Prior work is reported to improve training time significantly when curriculum-acquisition costs are neglected, particularly for large training data.
  • Consistency Analyses: Appendix figures test whether reported relationships remain consistent across datasets and architectures, including ensemble entropy and curriculum-related analyses.The cited figures cover all datasets and architectures for several relationships and use prediction depth to estimate ensemble prediction entropy.
  • Algorithmic Fairness: Label-noise filtering can disadvantage rare subgroups because mislabeled data and rare-subgroup examples may be processed like ambiguous inputs.The authors identify distinguishing these sources of difficulty as an open challenge for fairer, higher-accuracy algorithms.
  • Heteroscedastic Uncertainty: Uncertainty models typically treat example difficulty as one-dimensional, motivating estimates of where training points lie among the paper’s richer difficulty categories.The passage frames this as a plausible extension rather than an established result.

E Alternative Definitions for Prediction Depth

The paper discusses alternative ways to define or probe prediction depth and reports appendix analyses comparing its relationships across architectures, datasets, splits, and probe choices.

  • Alternative Definitions: Using the ground-truth label instead of the network’s final prediction would require separate rules for correctly and incorrectly classified validation points.The authors prefer their final-prediction definition because it is simpler than combining two rules.
  • Alternative Definitions: Defining prediction depth through networks of different depths could reveal a richer difficulty picture but would be expensive and architecture-dependent.Different depths can have different inductive biases, potentially inverting relative input difficulty.
  • Probe Choice: The study uses k-NN probes rather than logistic-regression probes because logistic regression completely separates CIFAR10 training data after VGG16’s first convolution.On a random 50% split, k-NN has lower accuracy until classes become entirely clustered.
  • Appendix Analyses: Appendix histograms compare mean prediction depth with mean learning iteration across training and validation splits for CIFAR10, CIFAR100, Fashion MNIST, and SVHN.The comparisons span ResNet18, VGG16, and MLP architectures; Fashion MNIST is noted as largely learned in the input layer.
  • Appendix Analyses: Additional figures examine prediction-depth relationships with input and output margins, consensus consistency, and internal processing across architectures and datasets.These analyses include correlation plots, split comparisons conditioned on consensus labels, and reproductions for CIFAR10 models.
  • Appendix Reproductions: The appendix reproduces reported behaviors for ResNet18, VGG16, and MLP on CIFAR10 and provides figure-specific consensus-versus-ground-truth comparisons.
  • Example Categories: The paper categorizes difficult examples into interpretable groups and shows that their confidence and accuracy trajectories differ as representations pass through hidden layers.Ambiguous examples without labels shift from ground-truth processing toward their consensus class, while ambiguous examples begin with low confidence.
Loading 2106.09647v2…