PaperlayerView exact revision on arXiv

Source-linked AI summary

Influence Functions in Deep Learning Are Fragile

Samyadeep Basu, Philip Pope, Soheil Feizi

arXiv:2006.14651v2cs.LGstat.ML

TL;DR

Influence functions are well understood in convex models but remain insufficiently studied in non-convex deep learning. This paper conducts a broad empirical study across datasets and neural-network architectures, finding that influence estimates are generally fragile and strongly dependent on model and evaluation choices.

  • Problem

    Influence functions are not well understood for deep-learning models with non-convex losses, despite their use in interpreting model predictions and other applications.

  • Method

    The paper performs a comprehensive empirical study across progressively complex architectures and datasets, using exact and approximate Hessian-based influence estimates alongside retraining-based ground truth.

  • Results

    Influence estimates are fairly accurate in shallow networks but often erroneous in deeper models, while stochastic inverse-Hessian approximations can also be erroneous, especially for deep networks.

  • Takeaways & Limitations

    Influence functions in deep learning are fragile, with estimate quality affected by architecture, depth, width, regularization, approximation method, and test-point selection.

  • Takeaways & Limitations

    The standard influence-function approximation assumes a convex loss and small parameter changes, while exact Hessian computation is feasible only for small models.

Abstract

from arXiv · show

Influence functions approximate the effect of training samples in test-time predictions and have a wide variety of applications in machine learning interpretability and uncertainty estimation. A commonly-used (first-order) influence function can be implemented efficiently as a post-hoc method requiring access only to the gradients and Hessian of the model. For linear models, influence functions are well-defined due to the convexity of the underlying loss function and are generally accurate even across difficult settings where model changes are fairly large such as estimating group influences. Influence functions, however, are not well-understood in the context of deep learning with non-convex loss functions. In this paper, we provide a comprehensive and large-scale empirical study of successes and failures of influence functions in neural network models trained on datasets such as Iris, MNIST, CIFAR-10 and ImageNet. Through our extensive experiments, we show that the network architecture, its depth and width, as well as the extent of model parameterization and regularization techniques have strong effects in the accuracy of influence functions. In particular, we find that (i) influence estimates are fairly accurate for shallow networks, while for deeper networks the estimates are often erroneous; (ii) for certain network architectures and datasets, training with weight-decay regularization is important to get high-quality influence estimates; and (iii) the accuracy of influence estimates can vary significantly depending on the examined test points. These results suggest that in general influence functions in deep learning are fragile and call for developing improved influence estimation methods to mitigate these issues in non-convex setups.

1 INTRODUCTION

The paper investigates when influence functions work in deep learning, where non-convexity makes their behavior poorly understood. Experiments find that influence estimates are fragile and depend strongly on network complexity, regularization, approximation methods, and test-point choice.

  • Motivation: Influence functions estimate how perturbing or removing training samples changes model parameters or test-time predictions more cheaply than repeated retraining.They are used to study individual or group training-sample effects.
  • Motivation: Deep-learning influence functions are difficult to assess because neural-network losses are non-convex, unlike the convex settings where influence functions are well-defined and generally accurate.Different perturbed-model initializations and increasing curvature can contribute to approximation error.
  • Study design: The study progressively evaluates influence estimates across increasingly complex neural networks and datasets, beginning with an exact-Hessian Iris case study and extending to MNIST, CIFAR, and ImageNet-scale settings.The experiments include architectures such as CNNs, ResNets, and other progressively more complex models.
  • Findings: Influence estimates are fairly accurate for shallow networks but often erroneous for deeper models, with network depth and width strongly affecting estimate quality.The authors partially attribute this pattern to increasing network curvature with depth.
  • Findings: Weight-decay regularization is important for high-quality estimates in certain architectures and datasets, while inverse-Hessian-vector-product stochastic approximations are especially erroneous in deep networks.The approximation error can contribute to low-quality influence estimates in deep models.
  • Findings: Influence-estimate quality varies substantially across test points, and even leave-one-out ground-truth estimates can be noisy on very large datasets such as ImageNet.The ImageNet-scale noise is partially attributed to model training and convergence.
  • Conclusion: Overall, the results characterize current influence functions in deep learning as sensitive and motivate more robust estimators for large-scale applications.The paper identifies architecture, depth, width, regularization, stochastic approximation, and test-point selection as influential factors.

2 RELATED WORKS

Related work uses influence functions to identify important training data and assess or manipulate model behavior across interpretability, auditing, bias detection, security, and group-influence tasks.

  • Applications: Influence functions identify important training samples for test-time predictions and help debug machine-learning models.They have also been applied to approximate datasets using subsets of data.
  • Applications: Prior applications include auditing test-prediction reliability, detecting bias in word embeddings, and crafting stronger data-poisoning attacks.These uses extend influence functions beyond standard interpretability settings.
  • Applications: Influence functions can also identify important groups of training samples rather than only individual examples.The related work cites group-influence applications alongside individual-sample analysis.

3 BASICS OF INFLUENCE FUNCTION

Influence functions estimate how infinitesimal training-sample reweighting changes model parameters and test loss using gradients and the Hessian. Removing a sample corresponds to up-weighting it by ϵ = −1/n, but the approximation assumes a strictly convex loss and positive-definite Hessian.

  • Influence functions approximate parameter changes from infinitesimal up-weighting of a training example using a first-order Taylor expansion around optimal parameters.The resulting influence depends on the inverse Hessian and the training-example gradient.
  • Removing a training point is represented by setting its up-weighting amount to ϵ = −1/n.
  • The influence on a test point’s loss is computed by combining the test-loss gradient, inverse Hessian, and training-example gradient.
  • I(z, zt)/n approximates the change in test-sample loss when training sample z is removed.
  • The approximation assumes a strictly convex loss and a positive-definite Hessian, while exact Hessian inversion can be expensive for large models.

4 WHAT CAN GO WRONG FOR INFLUENCE FUNCTIONS IN DEEP LEARNING?

First-order influence estimates depend on a small Taylor approximation gap, but non-convex deep networks can violate this condition. The gap grows with certain training choices and model scales, while inverse-Hessian-vector approximations can add errors in deep networks.

  • Influence estimates require the Taylor approximation gap to remain small, an assumption that is not generally satisfied for non-convex losses.
  • Without weight decay on Iris, parameter changes estimated by influence functions deviate substantially more from retraining results.
  • Increasing network depth or width raises the Taylor gap and reduces influence-estimate quality in the over-parameterized regime.
  • Stochastic inverse-Hessian-vector-product approximations are erroneous, especially for deep networks, and may contribute to low-quality estimates.

5 EXPERIMENTS

Experiments across progressively deeper and larger models show that influence-function accuracy is sensitive to regularization, test-point choice, architecture, and dataset scale. Shallow networks can be accurate, whereas deeper or highly parameterized settings often produce poor or noisy estimates.

  • Iris and small-network experiments: 0.97 versus 0.508 Spearman correlation shows that weight decay can substantially improve influence estimates in the Iris feed-forward network.Without weight decay, the Hessian is singular and requires damping; parameter changes estimated by influence functions also deviate more from retraining.
  • Iris and small-network experiments: Influence quality decreases with network depth, alongside increasing loss curvature and a larger gap between approximate and retrained parameter changes.The approximation error is particularly large beyond depth 5, and depth 8 produces a considerable decrease in Spearman correlation.
  • Iris and small-network experiments: 0.82 to 0.56 Spearman correlation indicates that increasing network width from 8 to 50 consistently reduces influence-estimate quality.
  • Iris and small-network experiments: Stochastic inverse-Hessian estimation has marginally lower Spearman correlation than exact computation across network depths.
  • Iris and small-network experiments: Weight decay improves estimates for selected CNN settings, but correlations fall for less influential training points and depend strongly on the regularization factor and test point.For the examined CNN, correlations begin decreasing when the weight-decay factor exceeds 0.01; across test points, top-influence correlations range from 0.92 to 0.38.
  • Experiments on larger architectures and datasets: Across MNIST and CIFAR-10, smaller shallow CNNs and LeNet generally perform better than deeper architectures, with test-point choice and architecture producing sizeable variation.CIFAR-100 ResNet-18 estimates are substantially poor, including for high-loss and median-loss test points.
  • ImageNet-scale experiments: ImageNet-scale influence estimates are highly imprecise, while leave-one-out retraining itself produces variable ground truths partly associated with incomplete model convergence.Two additional training epochs improve top-5 accuracy slightly to 92.336% (+0.034), while losses change by -0.679 for one test point and +0.066 for another.

6 DISCUSSION ON GROUND-TRUTH INFLUENCE

The study validates retraining from optimal parameters as a practical approximation for obtaining ground-truth influence estimates. It closely matches retraining from scratch while avoiding its computational cost.

  • Retraining from optimal parameters produces small parameter differences and similar correlation estimates compared with retraining from scratch.Retraining from scratch is computationally expensive even for moderately sized datasets and architectures.

7 CONCLUSION

The paper concludes that influence functions are generally fragile in deep learning. Their quality varies with model and evaluation choices, motivating more robust estimators for non-convex settings.

  • Experiments across Iris, MNIST, CIFAR-10, CIFAR-100, ImageNet, and multiple architectures show that influence-function quality is strongly affected by depth, width, architecture, weight decay, approximation, and test-point selection.
  • Influence estimates are fairly accurate in shallow architectures such as small CNNs and LeNet but often erroneous in very deep and wide models such as ResNet-50.
  • ImageNet-scale influence estimates are highly imprecise, supporting the need for robust influence estimators in non-convex deep-learning settings.

9 APPENDIX

Additional experiments show that influence quality declines with depth but improves with weight decay, while the evaluated test point strongly affects both quantitative and semantic results.

  • Influence correlations consistently decrease as ReLU network depth increases, alongside larger Hessian top eigenvalues and Taylor approximation gaps.These patterns partially explain why deeper networks produce poorer influence estimates.
  • For tanh networks, weight decay produces a small Taylor approximation gap, whereas omitting it produces a large gap and poor influence estimates.The tanh results mirror the reported ReLU findings.
  • Higher weight decay reduces curvature and improves influence correlations in deeper networks.For example, Spearman correlation improves from 0.47 at weight decay 0.03 to 0.72 at 0.075.
  • The choice of test point affects both influence-estimate quality and the semantic similarity of retrieved training samples.For CIFAR-10 test point 1479, only 3 of the top 5 samples are semantically similar; for point 7196, all 5 are.

9.4 RUNNING TIMES

The appendix describes influence computation as faster than retraining despite slowing with parameter count, then evaluates ImageNet test points and finds weak quantitative agreement despite some qualitative similarity.

  • Influence computation becomes relatively slow as parameter count grows but remains faster than retraining for every training example.The implementation first computes an inverse-Hessian vector product for the test point, then performs vector dot products for all training points.
  • ImageNet evaluation uses selected test points with different loss percentiles: kit fox at the 83rd percentile and gila monster at the 37th percentile.Their losses are 2.634 and 0.081, respectively.
  • Some highly influential ImageNet training samples are qualitatively similar to the test points, but the overall results remain weak quantitatively.The visualizations show qualitative matches alongside unrelated or questionable examples.
  • Figure 11 compares actual test-loss changes after removal and retraining with influence-estimated test-loss changes using Pearson and Spearman correlations.The caption reports low correlations for the selected ImageNet test points.

9.6 COMPUTING INVERSE-HESSIAN VECTOR PRODUCT

For large deep models, inverse-Hessian vector products are approximated rather than computed exactly, using conjugate-gradient or stochastic methods with regularization and tuning requirements.

  • Large over-parameterized networks avoid exact Hessian inversion by using Hessian-vector products with conjugate-gradient or stochastic estimation.These methods compute an approximate inverse-Hessian vector product.
  • Conjugate-gradient solves a quadratic optimization problem for t = H^-1v, where v is the test-point loss gradient.The optimization requires a positive-definite Hessian, which deep-network Hessians may not satisfy.
  • Adding damping λ to the Hessian makes it positive definite for conjugate-gradient optimization.The regularized matrix is H + λI.
  • Stochastic estimation samples training examples to estimate the Hessian recursively, avoiding a pass through every training sample.It requires tuning scaling γ, damping β, and recursion depth.
  • The experiments use stochastic estimation to compute inverse-Hessian vector products in large deep models.

9.7 EFFECT OF INITIALISATION AND OPTIMIZERS ON INFLUENCE ESTIMATES

Experiments examine initialization and training-sample selection, finding that influence quality depends on which samples and test points are evaluated and that explanations can be plausible without being consistently faithful.

  • Initialization effects are tested on small CNN and LeNet models trained on MNIST with a constant weight decay of 0.001.
  • Evaluating the most influential training points yields relevant Pearson and Spearman correlations across different sample counts, while evaluating the least influential points yields poor correlations.
  • Influence explanations for deep networks are sometimes plausible and sometimes not, including cases where a bird test point retrieves deer examples.Other test points produce many visually plausible explanations.
  • Influence functions are faithful when they answer what the model would have done if certain data were excluded.

9.10 CIFAR-100 INFLUENTIAL EXAMPLES

For CIFAR-100, influential examples can be either semantically dissimilar or similar to a test point. The supplied experiments also examine group influence, width, and parameter-change agreement.

  • For test point 7106, the top influential training samples are semantically dissimilar, whereas 4 of the top 5 for test point 2407 are semantically similar.
  • The experiments also compare parameter changes computed with influence functions against changes from retraining.
  • Additional experiments examine how network width relates to Spearman correlation in a one-layered network.
  • The study includes preliminary group-influence results whose estimates range from 0.01 to 0.18.
  • High-quality influence estimates are observed for certain networks with non-smooth ReLU activation functions, while their broader behavior remains a future research direction.
Loading 2006.14651v2…