PaperlayerView exact revision on arXiv

Source-linked AI summary

Why Normalizing Flows Fail to Detect Out-of-Distribution Data

Polina Kirichenko, Pavel Izmailov, Andrew Gordon Wilson

arXiv:2006.08545v1stat.MLcs.LG

TL;DR

Normalizing flows often fail to distinguish in-distribution from OOD data, raising the question of why exact-likelihood models perform poorly for this task. The paper analyzes their learned representations and coupling-layer transformations, showing that architectural changes and semantic high-level features improve OOD detection while standard flows favor generic graphical structure.

  • Problem

    The paper investigates why normalizing flows often assign higher likelihood to OOD data and why their failure mechanisms remain insufficiently understood.

  • Method

    The paper studies flow representations and image-to-latent transformations, then modifies coupling-layer architectures and trains flows on semantic high-level features.

  • Results

    Flows learn local pixel correlations and generic transformations rather than semantic content, while architectural changes and high-level semantic features improve OOD detection.

  • Takeaways & Limitations

    Flow properties that support high-fidelity image generation can hinder OOD detection by favoring graphical features over target-specific semantic structure.

  • Takeaways & Limitations

    The detailed study focuses on affine-coupling flows, and extending the conclusions to other flow types remains future work.

Abstract

from arXiv · show

Detecting out-of-distribution (OOD) data is crucial for robust machine learning systems. Normalizing flows are flexible deep generative models that often surprisingly fail to distinguish between in- and out-of-distribution data: a flow trained on pictures of clothing assigns higher likelihood to handwritten digits. We investigate why normalizing flows perform poorly for OOD detection. We demonstrate that flows learn local pixel correlations and generic image-to-latent-space transformations which are not specific to the target image dataset. We show that by modifying the architecture of flow coupling layers we can bias the flow towards learning the semantic structure of the target data, improving OOD detection. Our investigation reveals that properties that enable flows to generate high-fidelity images can have a detrimental effect on OOD detection.

1 Introduction

Normalizing flows can assign higher likelihood to OOD images than to training-distribution images, while learning generic transformations based on graphical features rather than semantics. Architectural changes and high-level semantic features improve OOD detection.

  • Motivation: Flows can assign higher likelihood to CelebA and SVHN than to ImageNet, despite being trained on ImageNet.This illustrates the central OOD-detection failure reported for RealNVP.
  • Motivation: The reasons for flows’ poor OOD detection remain insufficiently understood despite empirical progress.
  • Contributions: Flows learn image representations largely from local pixel correlations rather than semantic content, hindering detection of anomalous semantics.
  • Contributions: RealNVP coupling layers transform in-distribution ImageNet and OOD CelebA in the same way, revealing generic transformations across structured images.
  • Contributions: Changing coupling-layer architecture encourages target-specific transformations and improves OOD detection.
  • Contributions: Training flows on high-level features containing semantic information improves OOD detection.

2 Background

Normalizing flows model data through invertible transformations from a base latent distribution and provide exact likelihoods via the change-of-variables formula. Affine coupling layers make these transformations efficiently computable and stackable into flexible flow models.

  • Normalizing flows: Normalizing flows model a target distribution as an invertible transformation of a latent-space base distribution.
  • Normalizing flows: Flows commonly use a standard Gaussian latent distribution and train the invertible transformation by maximizing training-data log-likelihood.
  • Coupling layers: Affine coupling layers split inputs into disjoint parts and transform one part using scale and shift functions conditioned on the other.
  • Coupling layers: The coupling transformation is invertible and enables efficient Jacobian computation for likelihood evaluation.
  • Flows with coupling layers: Stacking coupling layers yields flexible flows including NICE, RealNVP, and Glow.
  • OOD detection: For likelihood-based OOD detection, inputs below a validation-derived likelihood threshold are identified as OOD.

3 Related Work

Prior work documents poor OOD detection by generative models and develops tests based on typicality, batch statistics, ensembles, background likelihoods, and image simplicity. This paper studies the inductive biases underlying these behaviors rather than treating flows as black-box density estimators.

  • Detection settings: OOD detection research distinguishes group anomaly detection, which labels batches, from point anomaly detection, which labels individual points.
  • Group anomaly detection: The typicality test cannot detect OOD data when its likelihood distribution resembles that of in-distribution data.
  • Existing approaches: Prior methods use batch-normalization statistics, generative-model ensembles, or image-to-background likelihood ratios to improve OOD detection.
  • This work: This paper investigates latent representations and image-to-latent transformations to connect OOD performance with model inductive biases.

4 Why flows fail to detect OOD data

The paper argues that flow inductive biases, more than the maximum-likelihood objective, determine which data receive high likelihood. Flows favor graphical regularities over semantic structure, causing high likelihood across structured datasets and undermining OOD detection.

  • Thesis: The maximum-likelihood objective has less influence on OOD detection than architectural inductive biases.
  • Thesis: Non-optimal likelihood solutions can redistribute probability mass outside the training data while preserving the objective value, leaving inductive biases to select among them.
  • Semantic structure: OOD detection requires inductive biases aligned with the semantic structure defining computer-vision datasets.
  • Flow biases: Normalizing flows are biased toward graphical properties such as local pixel correlations rather than semantic properties such as object identity.
  • Flow capacity: Under standard maximum likelihood, flows can assign high likelihood to most structured datasets even though they can distinguish datasets when explicitly trained to do so.

5 Flow latent spaces

Flow latent representations preserve graphical structure from both in-distribution and OOD images rather than encoding semantic content. This appearance-based representation limits likelihood-based OOD detection.

  • 5 Flow latent spaces: Latent representations encode simple graphical structure, such as recognizable input edges, rather than high-level semantic information.The correspondence between image coordinates and latent coordinates persists for both in-distribution and OOD inputs.
  • 5 Flow latent spaces: Averaging over 40 dequantization-noise samples makes the original image edges clearly visible in the latent representation.
  • 5 Flow latent spaces: Batch normalization in train mode produces visibly different latent structure for in-distribution and OOD inputs.For in-distribution data, no structure is visible, whereas OOD latent representations retain clear structure.
  • 5 Flow latent spaces: These visualizations explain why latent correlations can support prior OOD methods based on latent representations and batch-normalization statistics.

6 Transformations learned by coupling layers

Coupling layers achieve high likelihood by predicting masked pixels using local correlations and information propagated by earlier layers. These mechanisms work similarly on in-distribution and OOD images.

  • 6 Transformations learned by coupling layers: Coupling layers update masked pixels using scale and shift outputs predicted from the unmasked input.The likelihood increases when masked pixels are accurately predicted.
  • 6.1 Leveraging local pixel correlations: Local color correlations let checkerboard and channel-wise masks predict neighboring pixel values on both FashionMNIST and MNIST.These masks bias st-networks toward exploiting nearby-pixel similarity rather than dataset-specific semantics.
  • 6 Transformations learned by coupling layers: These mechanisms allow flows to predict masked pixels similarly across datasets, producing high likelihood for OOD data.
  • 6.2 Coupling layer co-adaptation: Horizontal masks reveal coupling-layer co-adaptation: later layers decode information about masked regions encoded by earlier layers.The second and third layers reconstruct masked regions almost ideally for both FashionMNIST and OOD MNIST inputs.
  • 6.2 Coupling layer co-adaptation: The authors hypothesize that co-adaptation also affects checkerboard and channel-wise masks when combined with local color correlations.
  • 6.2 Coupling layer co-adaptation: Lower st-network bottleneck dimensions improve OOD separation, while a larger dimension leaves more overlap with OOD likelihoods.In Figure 4, l = 50 and l = 10 separate FashionMNIST from MNIST better than l = 100.

7 Changing biases in flows for better OOD detection

The paper changes flow inductive biases to reduce reliance on local pixel correlations and coupling-layer co-adaptation. Masking, bottlenecks, and semantic embeddings improve the relative likelihood ranking of in-distribution data.

  • 7 Changing biases in flows for better OOD detection: Changing coupling-layer masks or st-network architecture improves likelihood-based OOD detection.
  • Changing masking strategy: Horizontal masks prevent direct use of neighboring pixels, while cycle masks require information to travel through three coupling layers.
  • Image embeddings: Semantic image embeddings make flows assign higher likelihood to in-distribution data, unlike raw pixels where SVHN receives the highest likelihood.
  • st-networks with bottleneck: A low-dimensional st-network bottleneck restricts reproduction of input pixels and reduces coupling-layer co-adaptation.The bottleneck is implemented by projecting to dimension l and back to the original input dimension.
  • 7 Changing biases in flows for better OOD detection: The proposed modifications do not completely eliminate higher likelihood for OOD data, but they improve the relative likelihood ranking for in-distribution data.

8 Out-of-distribution detection using image embeddings

Flows perform better at OOD detection when trained on high-level semantic image representations rather than raw pixels. Across datasets, these embeddings generally restore higher likelihood for in-distribution data.

  • 8 Out-of-distribution detection using image embeddings: Normalizing flows can detect OOD images when trained on high-level semantic representations instead of raw pixels.
  • 8 Out-of-distribution detection using image embeddings: For flows trained on SVHN and CelebA embeddings, both OOD datasets have lower likelihood and AUROC scores close to 100%.
  • 8 Out-of-distribution detection using image embeddings: The embedding dimension is only two times smaller than the original image dimension, so dimensionality alone does not explain the raw-image OOD failure.
  • 8 Out-of-distribution detection using image embeddings: 73% AUROC separates CIFAR-10 and SVHN embeddings, despite significant likelihood overlap.
  • 8 Out-of-distribution detection using image embeddings: On tabular UCI datasets with relatively high-level features, normalizing flows assign higher likelihood to in-distribution data.

9 Conclusion

The conclusion attributes flows’ OOD-detection failures to inductive biases that favor generic graphical features and local pixel correlations over target-specific semantic structure. It also limits the study’s scope primarily to affine coupling-layer flows while identifying extensions to other flow types as future work.

  • These inductive biases can guide flows toward high-likelihood representations that are useful for generation but detrimental to OOD detection.
  • Flows tend to learn generic graphical features and local pixel correlations rather than semantic structure specific to the training distribution.
  • The detailed analysis focuses on affine coupling-layer flows, while the authors hypothesize that conclusions extend to other flow types and identify full study of them as future work.
  • The appendix supplies additional experiment details, baseline samples, latent-variable resampling results, image-embedding and tabular-data results, and formal discussion.
  • Negative training can produce high likelihood on target data and low likelihood on a specified OOD dataset.
  • It also documents hyperparameters, likelihood histograms and AUROC scores, latent-visualization procedures, masking strategies, and coupling-layer visualizations.

A Maximum likelihood objective is agnostic to what data is OOD

The maximum-likelihood objective alone does not determine which data a normalizing flow treats as OOD. Different distributions can match the training data’s likelihood while assigning high likelihood to different image sets, including arbitrary OOD points.

  • Maximum likelihood alone does not define a normalizing flow’s OOD-detection performance; inductive biases determine which data receive high or low likelihood.
  • The flow-like distribution assigns high likelihood to smooth structured images, including images from SVHN and ImageNet, whereas the human-like distribution favors faces with small noise or brightness changes.
  • Two distributions can assign equal likelihood to CelebA training images while defining different high-probability regions in image space.
  • For any finite training set and arbitrary finite OOD set, a density can preserve training-point densities while assigning every OOD point density at least c.
  • The constructed density p′ integrates to one, coincides with p on training data, and redistributes mass toward an OOD neighborhood.

B Flows have capacity to distinguish datasets

Flows have enough capacity to distinguish target and OOD datasets when trained explicitly for separation, but standard likelihood training favors high likelihood for structured images broadly. Visualizations support that flows encode local image appearance rather than semantic dataset-specific structure.

  • Explicit separation: Flows can separate target and OOD datasets when explicitly trained to maximize target likelihood and minimize OOD likelihood.The objective uses a threshold c so OOD likelihood is reduced toward a finite level rather than −∞.
  • Explicit separation: The explicit-separation experiment succeeds across CIFAR-10 versus CelebA, CIFAR-10 versus SVHN, and FashionMNIST versus MNIST.In each pairing, OOD likelihood becomes very low while target likelihood is maximized, producing clear separation.
  • Baseline behavior: Standard RealNVP and Glow models consistently fail at OOD detection on image benchmarks.Their likelihood distributions assign similar or higher likelihood to OOD data than to target data.
  • Representation analysis: Latent representations preserve recognizable input edges, indicating correspondence between image coordinates and learned representations rather than semantic encoding.This pattern appears for RealNVP and Glow across FashionMNIST/MNIST and CelebA/SVHN visualizations.
  • Representation analysis: Coupling-layer visualizations provide empirical support for flows predicting masked pixels similarly on in-distribution and OOD inputs.The paper identifies local color correlations and coupling-layer co-adaptation as mechanisms enabling this behavior.
  • Architectural modification: Cycle-mask reduces the applicability of local-correlation and coupling-layer co-adaptation mechanisms in flow coupling layers.The strategy is introduced to bias representations toward better alignment with the target data’s structure.

K.3 UCI datasets

On two UCI physics datasets, flows show better OOD detection than on the reported image benchmarks, although train and OOD likelihoods still overlap.

  • Experimental setup: The experiment trains a model on one class of each binary UCI dataset and treats the other class as OOD data.The datasets are HEPMASS and MINIBOONE, whose classes represent background and signal.
  • Results: In-distribution data has higher average likelihood in all UCI setups, while test and OOD likelihood distributions overlap.This indicates partial rather than complete separation between in-distribution and OOD examples.
  • Results: AUROC ranges from 70% to 87%, substantially exceeding the image-benchmark results reported in prior work.The comparison is reported for the HEPMASS and MINIBOONE class-based OOD setups.
Loading 2006.08545v1…