PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks

John Bradshaw, Alexander G. de G. Matthews, Zoubin Ghahramani

arXiv:1707.02476v1stat.ML

TL;DR

The paper addresses whether neural networks can retain strong representations while providing better uncertainty estimates and robustness beyond their training distribution. It trains Gaussian-process output layers on deep networks and evaluates classification, adversarial examples, and domain-shift transfer, finding more robust uncertainty behavior and competitive predictive performance. The authors conclude that GPDNNs are promising where calibrated uncertainty matters, while noting architecture- and setting-specific limitations.

  • Problem

    DNNs have strong representations but often capture uncertainty poorly, whereas GPs have better uncertainty behavior but limited scale and representational power on complex image data.

  • Method

    The paper trains Gaussian processes on top of CNN representations end-to-end and evaluates GPDNNs on classification, adversarial attacks, and transfer to new domains.

  • Results

    GPDNNs provide competitive classification, appear less susceptible to FGSM attacks, and show better log likelihoods with more uncertain outputs under domain shift.

  • Takeaways & Limitations

    GPDNNs are promising for applications where well-calibrated uncertainties are important because they can avoid overconfident predictions in unfamiliar regions.

  • Takeaways & Limitations

    The reported properties have scope boundaries: architecture families may scale differently with inducing-point number, and CIFAR-10 FGSM results do not exactly match other findings.

Abstract

from arXiv · show

Deep neural networks (DNNs) have excellent representative power and are state of the art classifiers on many tasks. However, they often do not capture their own uncertainties well making them less robust in the real world as they overconfidently extrapolate and do not notice domain shift. Gaussian processes (GPs) with RBF kernels on the other hand have better calibrated uncertainties and do not overconfidently extrapolate far from data in their training set. However, GPs have poor representational power and do not perform as well as DNNs on complex domains. In this paper we show that GP hybrid deep networks, GPDNNs, (GPs on top of DNNs and trained end-to-end) inherit the nice properties of both GPs and DNNs and are much more robust to adversarial examples. When extrapolating to adversarial examples and testing in domain shift settings, GPDNNs frequently output high entropy class probabilities corresponding to essentially "don't know". GPDNNs are therefore promising as deep architectures that know when they don't know.

1 Introduction

The paper combines DNN representations with GP output layers to address weak uncertainty calibration, adversarial brittleness, and domain-shift detection. GPDNNs are trained end-to-end and evaluated for classification, adversarial robustness, and transfer testing.

  • Motivation: DNNs provide strong representations but often fail to capture uncertainty, while GPs face scale and representational limitations on complex image data.Regular GPs require costly covariance operations and simple kernels do not represent high-dimensional image relationships as effectively as DNNs.
  • Training: The paper trains GPDNNs end-to-end with scalable variational methods using GPflow and TensorFlow.The implementation back-propagates through the GP components, including the Cholesky decomposition.
  • Evaluation: Experiments examine image classification, adversarial robustness, and transfer testing under domain shift.The authors report competitive classification accuracy, greater apparent resistance to adversarial examples, and more robust behavior on challenging new domains.
  • Model motivation: GPDNNs place Gaussian processes on top of deep neural networks to combine learned representations with GP probability estimates.The models use a base CNN whose features feed either a softmax, a linear classifier with softmax, or a GP.
  • Architecture: The hybrid architectures use the same base CNN family while varying the final layer, with GPDNN architecture C differing from regular DNN architectures A and B.Architecture B adds an extra fully connected layer, whereas architecture C uses a GP with as many latent functions as classes.
  • Scope: The authors expect the reported promising properties may also apply to related hybrid models, while noting a scaling tradeoff in the number of inducing points.This scope statement is tied to the architecture comparison and is presented as an expectation rather than a demonstrated result.

2 Classification

The classification experiments compare GPDNNs with neural-network alternatives on MNIST and CIFAR-10 while varying the available training data. GPDNNs generally perform best or marginally better, especially in low-data MNIST settings, although a longer-trained NN is strongest on one CIFAR-10 condition.

  • Experimental setup: The experiments evaluate error rates and log likelihoods as the amount of training data varies on MNIST and CIFAR-10.MNIST uses a fixed 5000-image validation set, while CIFAR-10 compares models across training proportions.
  • MNIST: GPDNNs generally achieve lower error rates and higher log likelihoods than NN models on MNIST, with the largest gap under limited data.The comparison includes regular architectures and GPDNNs with linear and RBF kernels.
  • MNIST: The MNIST RBF-kernel GPDNN has better log likelihoods than the linear-kernel GPDNN, while their error rates are similar.The passage attributes this possibility to the RBF kernel’s extrapolation behavior, but the supplied text truncates the explanation.
  • CIFAR-10: GPDNNs are marginally better than regular NNs in both accuracy and log likelihood on CIFAR-10 across the reported comparisons.The longest-training NN instead has the lowest error when trained on 10% of CIFAR-10’s training set.
  • CIFAR-10: CIFAR-10 hybrid networks train for 500 epochs and switch from architecture B to C, whereas regular NN training duration varies by condition.With smaller dataset proportions, the number of training steps decreases because epochs contain fewer batches.

3 Robustness to adversarial examples

The paper evaluates GPDNN robustness against non-targeted FGSM and L2 adversarial attacks. GPDNNs show slower error degradation, more uncertainty on transferred adversarial examples, and greater resistance to successful L2 attacks.

  • Attack setting: Adversarial examples are small perturbations that can cause incorrect predictions and transfer across architectures and machine-learning methods.The paper considers non-targeted attacks, which seek any changed class prediction rather than a specified target.
  • FGSM: GPDNN error increases much more slowly as FGSM perturbation size grows on MNIST.The comparison uses the NN (SC) architecture A and the architecture C GPDNN classifier.
  • FGSM: FGSM adds a perturbation based on the sign of the loss gradient, exploiting changes in feature space that can move inputs across classification boundaries.The loss corresponds to the negative log likelihood for both GPDNNs and regular neural networks.
  • FGSM transfer: On adversarial examples attacking NN (SC), GPDNNs outperform NN (SC) in error and log likelihood and perform comparably to the deeper NN (DC).For large perturbations, GPDNN predictive entropy approaches the entropy of a uniform distribution, indicating reduced confidence in unknown regions.
  • L2 attack: In the L2 attack experiment, 381 of 1000 GPDNN attacks failed, while successful attacks required perturbations averaging 0.529 greater in Euclidean distance.The perturbation distance is measured on images normalized between -1 and 1.
  • Overall robustness: The authors conclude that GPDNNs are less susceptible to FGSM and require more failed or larger successful perturbations under the L2 attack.The FGSM conclusion is qualified as applying at least on simple datasets.

4 Transfer testing

The transfer-testing study examines classifiers trained on MNIST and evaluated on other image datasets without fine-tuning. GPDNNs retain substantially better log likelihoods than regular NNs on harder datasets and produce less overconfident predictions.

  • Uncertainty: Regular NNs suffer from overconfidence, whereas GPDNN predictions are closer to the ideal “I don’t know” log likelihood of -2.3.This supports evaluating whether uncertainty adjusts when models encounter domains different from their training distribution.
  • Evaluation setup: Transfer testing evaluates classifiers trained on MNIST when used on other image datasets without fine-tuning.The datasets are converted to grayscale where needed and downsampled to MNIST image size.
  • Evaluation setup: The transfer datasets include MNIST, ANOMNIST, Semeion, and SVHN, with generalization from MNIST to the other three posed as the challenge.These datasets are illustrated in Figure 7.
  • Models: The comparison includes NN and GPDNN models built from small and deeper base architectures.The models compared are NN (SC, arch A), GPDNN (SC, arch C), NN (DC, arch B), and GPDNN (DC, arch C).
  • Results: Semeion and SVHN are much harder datasets, while NN (DC) achieves higher accuracy and the two small-architecture models have little accuracy difference.The deeper GPDNN performs worse than the small GPDNN in log likelihood on the two harder datasets.
  • Results: GPDNNs have substantially better log likelihoods than regular NNs on the two harder test datasets.The reported comparison uses the log likelihood of a uniform ten-class predictor, -2.3, as an “I don’t know” reference.

5 Conclusions

GPDNNs combine CNN representations with GP-like uncertainty behavior, showing robustness benefits in classification, adversarial testing, and domain shift. The authors interpret these results as evidence that RBF-kernel GPDNNs avoid overconfident extrapolation.

  • Conclusions: GPDNNs use CNN representations while retaining better-calibrated uncertainties and competitive classification performance.The authors report that these models combine powerful learned representations with properties associated with Gaussian processes.
  • Conclusions: GPDNNs are less susceptible to FGSM attacks and often scale predictive entropy with their error rates on adversarial examples.
  • Conclusions: GPDNNs achieve better log likelihoods on new domains, suggesting that they do not extrapolate poorly.
  • Conclusions: In a half-moon example, the RBF-kernel GPDNN does not overconfidently extrapolate as far as the compared models.The example contrasts neural-network, GP, and GPDNN decision boundaries using linear and RBF kernels.

B.2.1 Small CNN (SC) architectures

The MNIST small-CNN architectures use 100 inducing points for the GP component, and the SC model family is documented in Table 2.

  • B.2.1 Small CNN (SC) architectures: The GP component of the GPDNN models uses 100 inducing points.
  • B.2.1 Small CNN (SC) architectures: Table 2 presents the SC family of models used on MNIST.
  • B.2.1 Small CNN (SC) architectures: The architecture description is organized around the small-CNN model family used for MNIST experiments.

B.2.2 DD-style CNN (DC) architectures

The MNIST DD-style CNN and DenseNet architectures define the CNN backbones used with GPDNN models, including their regularization, optimization, and inducing-point settings.

  • B.2.2 DD-style CNN (DC) architectures: The DD-style CNN uses dropout rate 0.5 on its fully connected layer and is trained for 50 epochs with batch size 128 using ADAM.
  • B.2.2 DD-style CNN (DC) architectures: The GPDNN version of the DD-style CNN uses 100 inducing points.
  • B.2.2 DD-style CNN (DC) architectures: The DenseNet architecture used for CIFAR-10 is documented in Table 4, while the DD-style CNN architecture is documented in Table 3.
  • B.2.2 DD-style CNN (DC) architectures: The DenseNet backbone consists of convolutional blocks and transition layers with pooling, while dropout and batch normalization are turned off for GPDNN training.L2 weight regularization is applied to the CNN weights.
  • B.2.2 DD-style CNN (DC) architectures: GPDNN models are trained with ADAM, whereas softmax-top models are trained via SGD with a learning-rate decay regime.

B.3.2 DD-style CNN (DC) architecture

The CIFAR-10 DD-style CNN experiments compare NN and GPDNN models under adversarial-robustness settings. The results favor GPDNNs on error rates and learned robustmax calibration, while the authors qualify the FGSM explanation for deeper nonlinear networks.

  • B.3.2 DD-style CNN (DC) architecture: The CIFAR-10 adversarial-robustness experiments use a shallower DD-style CNN adapted from Papernot et al. and Carlini and Wagner.
  • B.3.2 DD-style CNN (DC) architecture: The GPDNN uses 100 inducing points, and experiments learn the robustmax β parameter because the models have lower accuracy.Both models are trained with ADAM for 100 epochs and batch size 128.
  • B.3.2 DD-style CNN (DC) architecture: GPDNNs have slightly smaller error rates than the NN model on the CIFAR-10 test set.
  • B.3.2 DD-style CNN (DC) architecture: The GPDNN with learned β is the best model in the reported comparison.
  • Why might the FGSM sometimes fail with classifiers of greater nonlinearity?: With an RBF kernel, FGSM may reduce the current-class likelihood without moving the example toward the other class, leaving it correctly classified.The authors caution that this explanation may not extend reliably to deeper networks with highly nonlinear decision boundaries, which may explain differing CIFAR-10 results.

C.1.2 MNIST results – applying both models to the adversarial examples generated attacking the GPDNN model

The MNIST evaluation applies both models to FGSM examples generated by attacking the GPDNN model, using error rates, log likelihoods, and predictive entropies as metrics.

  • Figures 10–12 report error rates, log likelihoods, and predictive entropies on FGSM examples attacking the GPDNN model.The same metrics are evaluated for both models on these transferred adversarial examples.
  • The evaluation uses adversarial examples generated by attacking the GPDNN rather than the NN.

C.1.3 CIFAR-10 results

On CIFAR-10, FGSM attacks affect both models with small perturbations, while the GPDNN generally retains better error, likelihood, and uncertainty behavior; cross-model transfer is poor.

  • FGSM robustness: The GPDNN generally has smaller increases in error rate, smaller log-likelihood drops, and higher predictive entropies than the NN under FGSM attacks.The learned β GPDNN performs best in the comparison shown in Figure 13.
  • FGSM robustness: The learned robustmax β improves GPDNN performance in the CIFAR-10 experiments.
  • Interpretation: The FGSM attack is more effective against the CIFAR-10 GPDNN than against the MNIST GPDNN.The authors associate this difference with the more complicated input space and weaker feature clustering.
  • FGSM robustness: Both models are fragile: small FGSM perturbations can fool them, although the GPDNN performs better.The relevant perturbation scale is smaller than in the MNIST experiments.
  • Transfer testing: Adversarial examples transfer poorly between the NN and GPDNN on CIFAR-10.Examples generated against one model do not work well when evaluated on the other.

C.2.2 MNIST results

MNIST L2 attacks show substantial resistance of the GPDNN, with failed attacks, larger required perturbations, and low transferability across models.

  • Attack success: 381 out of 1000 L2 attacks against the GPDNN failed, whereas all attacks against the regular NN succeeded.The authors used default attack parameters and note that tuning might make the attack more effective.
  • Transferability: L2 adversarial examples transfer poorly between the NN and GPDNN on MNIST.Table 7 evaluates examples generated against each model on both models, excluding failed GPDNN attacks.
  • Perturbation distances: The GPDNN requires greater perturbation distances than the NN in the reported pairwise comparisons.The histograms compare perturbation differences across models; failed attacks produce an artificial clump around 25 on GPDNN plots.
  • Perturbation distances: The deeper NN (DC) requires larger perturbations than the shallower NN (SC), but remains less robust than the GPDNN.
  • Qualitative examples: Figures 20–23 provide adversarial-image examples and corresponding model predictions for the MNIST attacks.

C.2.3 CIFAR-10 Results

On CIFAR-10, L2 attacks fail more often against the GPDNN, transfer poorly between models, and require larger perturbations for the GPDNN than for the NN.

  • Attack success: 207 CIFAR-10 attacks against the GPDNN failed, while all attacks against the regular NN succeeded.The transfer table excludes failed GPDNN attacks.
  • Transferability: L2 adversarial examples have poor transferability between the GPDNN and NN on CIFAR-10.The models are evaluated on examples generated by attacking each model.
  • Perturbation distances: The attacker traveled 0.071 farther on average against the GPDNN than against the NN, excluding failed attacks.Images were normalized between 0 and 1 and had three channels.
  • Qualitative examples: Figures 25–27 show original CIFAR-10 images and adversarial examples, including attacks with distances 0.19 versus 0.32 for an airplane and 0.04 versus 0.08 for a deer.The GPDNN examples are visually subtle in the illustrated cases.
  • Qualitative examples: Figures 28–29 compare predictions from the GPDNN and NN on original images and adversarial examples generated against each model.
Loading 1707.02476v1…