PaperlayerView exact revision on arXiv

Source-linked AI summary

Do Deep Generative Models Know What They Don't Know?

Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, Balaji Lakshminarayanan

arXiv:1810.09136v3stat.MLcs.LG

TL;DR

The paper asks whether deep generative-model densities can identify inputs similar to the training distribution. It evaluates multiple generative-model classes and studies the phenomenon analytically in flow models, finding that out-of-distribution inputs can receive higher likelihoods and urging caution in using density alone for detection.

  • Problem

    Deep generative models are expected to identify out-of-distribution inputs by assigning higher density to training-distribution data, but the paper investigates whether this expectation holds.

  • Method

    The paper evaluates PixelCNNs, VAEs, and flow-based models across image-dataset pairings, then analyzes constant-volume flows using data statistics and model curvature.

  • Results

    Across several datasets and model classes, out-of-distribution data can receive higher likelihood than training-distribution data; constant-volume-flow analysis explains the gap using data location, variance, and curvature.

  • Takeaways & Limitations

    Likelihoods from deep generative models alone cannot identify the training set or inputs similar to it, so their use for out-of-distribution detection warrants caution.

  • Takeaways & Limitations

    The analysis is constrained to specific flow settings when asserting persistence across parameter values and latent densities, including constant-volume Glow with log-concave latent densities.

Abstract

from arXiv · show

A neural network deployed in the wild may be asked to make predictions for inputs that were drawn from a different distribution than that of the training data. A plethora of work has demonstrated that it is easy to find or synthesize inputs for which a neural network is highly confident yet wrong. Generative models are widely viewed to be robust to such mistaken confidence as modeling the density of the input features can be used to detect novel, out-of-distribution inputs. In this paper we challenge this assumption. We find that the density learned by flow-based models, VAEs, and PixelCNNs cannot distinguish images of common objects such as dogs, trucks, and horses (i.e. CIFAR-10) from those of house numbers (i.e. SVHN), assigning a higher likelihood to the latter when the model is trained on the former. Moreover, we find evidence of this phenomenon when pairing several popular image data sets: FashionMNIST vs MNIST, CelebA vs SVHN, ImageNet vs CIFAR-10 / CIFAR-100 / SVHN. To investigate this curious behavior, we focus analysis on flow-based generative models in particular since they are trained and evaluated via the exact marginal likelihood. We find such behavior persists even when we restrict the flows to constant-volume transformations. These transformations admit some theoretical analysis, and we show that the difference in likelihoods can be explained by the location and variances of the data and the model curvature. Our results caution against using the density estimates from deep generative models to identify inputs similar to the training distribution until their behavior for out-of-distribution inputs is better understood.

1 INTRODUCTION

The paper questions whether deep generative-model densities can reliably identify inputs from the training distribution. Across several model classes and datasets, it finds higher likelihoods for visually distinct out-of-distribution data and analyzes this behavior in flow models.

  • Motivation: Deep generative models are proposed for detecting novel inputs because modeling p(x) may reveal whether an input differs from the training distribution.The motivation includes anomaly detection, uncertainty estimation, covariate-shift detection, active learning, reinforcement learning, and transfer learning.
  • Motivation: Autoregressive and invertible models are especially attractive because they provide exact marginal-likelihood computation without approximate inference.
  • Main finding: Models trained on CIFAR-10—including VAEs, autoregressive models, and flows—assign higher density to visually distinct SVHN images than to their training data.The paper reports similar findings for models trained on CelebA and ImageNet.
  • Main finding: The same out-of-distribution likelihood behavior appears across multiple deep generative-model classes and image datasets.The reported datasets include CIFAR-10, SVHN, CelebA, and ImageNet.
  • Analysis: Restricting flow models to constant-volume transformations still permits the likelihood differences to be explained by data location, variance, and model curvature.

2 BACKGROUND

The background formalizes density modeling through likelihood maximization and motivates neural density models for complex, high-dimensional data. It then introduces change-of-variables flows, whose Jacobian volume terms determine how densities transform.

  • Notation: The dataset consists of independently and identically distributed observations from an unknown population p*(x), while p(x; θ) denotes the model density.
  • Density estimation: Maximum-likelihood training selects parameters that maximize the data log density because the entropy term in the KLD is fixed.
  • Density estimation: A density value p(x; θ) is not itself a probability; probabilities require integrating the density over a region.
  • Training objective: Input noise produces a lower-bound training objective, and every reported model is trained with input noise.
  • Neural generative models: The paper studies PixelCNNs, VAEs, and invertible flow-based models, using neural networks to parameterize densities for complex, high-dimensional data.
  • Flow models: Change-of-variables models map data x to latent z through a diffeomorphism and compute p(x) using an auxiliary density and volume-element correction.
  • Flow models: Flow samples are generated by applying the inverse transform to latent samples drawn from the fixed auxiliary density.
  • Affine coupling layers: Affine coupling layers split the input, transform one half using neural-network scaling and translation, and pass the other half through unchanged.Stacked coupling layers produce the latent representation; permutations between layers ensure broader coordinate interaction.

3 MOTIVATING OBSERVATIONS

Glow assigns systematically higher likelihood to several out-of-distribution datasets than to data from the training distribution. The effect persists across dataset pairings and appears in PixelCNNs and VAEs as well.

  • Experimental setup: Glow is trained on FashionMNIST and CIFAR-10 to test whether likelihood distinguishes in-distribution from out-of-distribution inputs.The experiment uses a smaller version of the Glow architecture that fits on one GPU.
  • FashionMNIST vs MNIST: More than 1 bit lower BPD is obtained for MNIST than for FashionMNIST training and test data under a FashionMNIST-trained Glow.The histogram shows MNIST likelihoods systematically shifted higher, ruling out an outlier-driven average.
  • CIFAR-10 vs SVHN: 1 bit lower BPD is obtained for SVHN than for both CIFAR-10 training and test data under a CIFAR-10-trained Glow.The corresponding histogram shows systematically higher SVHN likelihoods rather than an effect caused by outliers.
  • Additional datasets: Glow trained on CelebA assigns higher likelihood to SVHN, while ImageNet-trained Glow assigns higher likelihood to SVHN, CIFAR-10, and CIFAR-100.The difference is drastic for SVHN and modest for the two CIFAR splits; the phenomenon is not symmetric for CIFAR-10 and SVHN.
  • Experimental setup: The smaller Glow model achieves CIFAR-10 BPD 3.46 versus 3.35 for the reference model.
  • Other generative models: The higher-likelihood out-of-distribution pattern appears in PixelCNNs and VAEs as well as flow-based models.Red histogram bars are shifted right for both FashionMNIST-versus-MNIST and CIFAR-10-versus-SVHN comparisons.

4 DIGGING DEEPER INTO THE FLOW-BASED MODEL

The authors isolate why flow-based models assign higher likelihood to out-of-distribution data, finding that the volume term dominates and that the effect persists under constant-volume transformations.

  • Motivation: Flow-based models are analyzed because they provide exact marginal likelihoods and Jacobian constraints that simplify experimental control.The investigation narrows from PixelCNNs, VAEs, and Glow to invertible models.
  • Likelihood decomposition: SVHN receives higher likelihood than CIFAR-10 because its volume-element evaluations are higher while its latent-density evaluations are only slightly lower.The volume term dominates the change-of-variables objective.
  • Likelihood decomposition: The volume term rewards increasing sensitivity to input perturbations through the Jacobian determinant, contrasting with derivative-based regularization that penalizes sensitivity.This identifies a tension between likelihood optimization and established Jacobian-regularization objectives.
  • Constant-volume analysis: Constant-volume Glow still assigns SVHN higher likelihood than CIFAR-10, showing the phenomenon does not require input-dependent volume changes.The model uses translation-only affine layers while retaining 1 × 1 convolutions.
  • Additional experiments: Constant inputs achieve the highest likelihood in NVP-Glow, reaching 0.589 BPD for CIFAR-10.Random inputs are included as a comparison in the appendix experiment.

5 SECOND ORDER ANALYSIS

The second-order analysis explains higher out-of-distribution likelihood through data moments and model curvature. Empirically, smaller variance and graying increase likelihood, matching the proposed explanation.

  • Second-order approximation: The analysis assumes equal means for SVHN and CIFAR-10, which the authors report is supported by their empirical per-dimension means.All SVHN means fall within CIFAR-10’s empirical range, with a maximum difference below 38 pixel values.
  • Second-order approximation: The expected log-likelihood gap can be approximated by a trace involving model curvature and the difference between the two distributions’ covariance matrices.Under equal means, the approximation is 1/2 Tr{∇2_x0 log p(x0; θ)(Σq − Σp*)}.
  • CV-Glow analysis: For flow-based models, the second-order expression is used to connect likelihood differences to curvature and the data’s second moment.The CV-Glow analysis simplifies the derivatives further under a spherical latent density.
  • CV-Glow analysis: The analysis predicts higher SVHN likelihood than CIFAR-10 for CV-Glow and remains non-negative across parameter settings under the stated conditions.The prediction agrees with the nearly unchanged likelihood gap from ensembles and throughout training.
  • Data statistics: SVHN has roughly the same mean as CIFAR-10 but smaller variance, placing it inside the CIFAR-10 distribution and producing higher likelihood.Across ImageNet comparisons, smaller variance likewise corresponds to higher likelihood.
  • Data statistics: Graying images increases likelihood by shrinking pixel variance, and reducing latent-representation variance has the same effect.The effect is shown for both CIFAR-10 and SVHN.

6 RELATED WORK

The paper situates its contribution within work showing that generative-model likelihood is not necessarily a reliable indicator of sample quality or anomaly status. Unlike prior adversarial-input studies, it examines calibration on ordinary out-of-distribution datasets and focuses on explaining the phenomenon.

  • Prior evaluation work showed that high likelihood is neither sufficient nor necessary for visually satisfying generated samples.
  • This paper studies calibration on common out-of-distribution datasets rather than adversarial inputs crafted against a particular model.
  • Concurrent studies also found PixelCNN++ unreliable for outlier detection, whereas this work investigates how and when the phenomenon arises.

7 DISCUSSION

The discussion concludes that likelihood comparisons alone cannot identify training-distribution membership, motivating caution and further investigation of high-dimensional generative-model likelihoods.

  • Likelihood comparisons alone cannot identify the training set or inputs similar to it.
  • The CIFAR-10 versus SVHN phenomenon persists for constant-volume Glow across parameter values and log-concave latent densities.
  • Alternative metrics and modified training procedures can enable out-of-distribution detection, but the broader likelihood problem remains open.
  • The authors urge caution when deploying generative models on out-of-training-distribution inputs or in unprotected user-facing systems.

A ADDITIONAL IMPLEMENTATION DETAILS

The appendix documents Glow, PixelCNN, VAE, and constant-volume Glow configurations, along with optimization and regularization choices used in the experiments.

  • Glow architecture details were introduced as practical choices, including 1 × 1 convolutions replacing discrete permutations.
  • Glow used dataset-specific coupling-block depths, Highway networks, squeezing or multi-scale structure, and shallower networks than the original Glow configuration.
  • Training generally used RMSProp or Adam with scheduled learning-rate decay, batch size 32, a standard Gaussian prior, and CIFAR-specific L2 regularization.
  • Constant-volume Glow used additive coupling blocks, leaving convolutional volume change dependent on weights rather than inputs.
  • The VAE experiments followed published CIFAR configurations and specified a separate encoder-decoder architecture for FashionMNIST.

B RESULTS ILLUSTRATING ASYMMETRIC BEHAVIOR

The appendix analyzes asymmetric likelihood behavior and its optimization mechanisms, showing how dimensionality, parametrization, input structure, and model averaging affect likelihoods.

  • Glow trained on SVHN or MNIST can assign lower likelihood to CIFAR-10 or FashionMNIST, demonstrating asymmetry relative to the reverse comparisons.
  • The change-of-variables objective combines latent log density with a log-Jacobian volume term whose bounds scale with dimensionality and Lipschitz constant.
  • The latent distribution’s peakedness must keep pace with the Lipschitz constant, especially as dimensionality increases, for both objective terms to contribute comparably.
  • The concentration analysis links latent concentration to the data variance and Lipschitz constant, while unconstrained optimization rewards increasing derivatives without directly enforcing latent concentration.
  • Sigmoid scale parametrization bounds the volume contribution above and changes its dimensionality behavior, but does not eliminate out-of-distribution overconfidence.
  • Constant inputs received the highest likelihood in NVP-Glow tests, while ensembles did not significantly change the relative CIFAR-10 versus SVHN difference.

G DERIVATION OF CV-GLOW’S LIKELIHOOD DIFFERENCE

The CV-Glow derivation isolates how constant-volume transformations affect likelihood, separating latent-density and curvature-related terms while removing input-dependent volume effects.

  • Constant-volume transformation: The CV-Glow volume element is constant across inputs, so it drops from the likelihood equation.This permits analysis of the volume element independently of input x0.
  • Ensemble analysis: The ensemble of Glow models remains non-robust to the likelihood phenomenon when trained on CIFAR-10 and tested on SVHN.Figure 9 presents a histogram of ensemble log-likelihoods.
  • Likelihood decomposition: The derivation separates the change-of-variables objective into latent log probability and volume-related terms.The first term is the log probability under the latent distribution.
  • Derivative structure: For f composed of translations and 1 × 1 convolutions, partial derivatives involve the convolutional operations because translation derivatives are all ones.The derivation indexes spatial, channel, flow, and kernel dimensions when forming these derivatives.
  • Final simplification: The derivation simplifies the full equation for constant-volume Glow and then assumes diagonal covariance matrices so only diagonal Hessian elements contribute.The diagonal assumption applies to both Σq and Σp∗.

H HISTOGRAM OF DATA STATISTICS

This section presents histograms of per-dimensional data means and code means, alongside dataset-statistics details for the analyzed image collections.

  • Data statistics: Per-dimensional means are summarized with a histogram across the analyzed data dimensions.The statistic is defined using µd = 1 N.
  • Dataset statistics: The datasets include 32 × 32 × 3 images from SVHN, CIFAR-10, CIFAR-100, CelebA, and ImageNet.Pixel values are converted from the 0-255 scale to the 0-1 scale by dividing by 256.
  • Code statistics: CV-Glow codes are analyzed with histograms of their means in the left column.Figure 11 is specifically described as an analysis of codes obtained using CV-Glow.

I RESULTS ILLUSTRATING EFFECT OF GRAYING ON CODES

The appendix examines how graying changes CV-Glow representations and presents samples from the models used in the analysis.

  • Graying analysis: Figure 12 shows the effect of graying on CV-Glow codes.The figure reports code mean, standard deviation, and norm.
  • Variance effect: Reducing the variance of latent representations has the same likelihood effect as the analyzed code transformation.This effect is shown in Figure 12 in the appendix.
  • Model samples: Figure 13 presents samples from the CV-Glow models used for the analysis.
Loading 1810.09136v3…