PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Likelihoods with Conditional Normalizing Flows

Christina Winkler, Daniel Worrall, Emiel Hoogeboom, Max Welling

arXiv:1912.00042v2cs.LGcs.CVstat.ML

TL;DR

High-dimensional structured prediction needs conditional distributions that capture correlations and multimodality without extensive modeling choices. The paper studies conditional normalizing flows, including binary dequantization, and reports competitive performance across super-resolution and binary segmentation.

  • Problem

    High-dimensional structured prediction is poorly served by independent per-dimension likelihoods when outputs contain correlated structure and multimodality.

  • Method

    The paper learns conditional likelihoods with CNFs and proposes variational dequantization for binary random variables.

  • Results

    Competitive performance is demonstrated against competing methods in super-resolution and binary image segmentation using likelihood and conventional metrics.

  • Takeaways & Limitations

    CNFs provide a probabilistic framing for supervised prediction tasks across super-resolution and binary image segmentation.

  • Takeaways & Limitations

    Adversarial methods can have unstable training and be difficult to evaluate quantitatively.

Abstract

from arXiv · show

Normalizing Flows (NFs) are able to model complicated distributions p(y) with strong inter-dimensional correlations and high multimodality by transforming a simple base density p(z) through an invertible neural network under the change of variables formula. Such behavior is desirable in multivariate structured prediction tasks, where handcrafted per-pixel loss-based methods inadequately capture strong correlations between output dimensions. We present a study of conditional normalizing flows (CNFs), a class of NFs where the base density to output space mapping is conditioned on an input x, to model conditional densities p(y|x). CNFs are efficient in sampling and inference, they can be trained with a likelihood-based objective, and CNFs, being generative flows, do not suffer from mode collapse or training instabilities. We provide an effective method to train continuous CNFs for binary problems and in particular, we apply these CNFs to super-resolution and vessel segmentation tasks demonstrating competitive performance on standard benchmark datasets in terms of likelihood and conventional metrics.

1 INTRODUCTION

High-dimensional conditional distributions require choices about factorization, correlations, modality, tails, likelihoods, and quantization. The paper proposes conditional normalizing flows to learn these distributions with fewer modeling choices and evaluates them on super-resolution and vessel segmentation.

  • High-dimensional conditional-distribution learning requires choices about factorization, correlations, modality, tails, likelihoods, and quantization.
  • Normalizing flows transform a simple prior through a learnable invertible mapping and change-of-variables formula to model complex distributions.
  • Normalizing flows support efficient inference and sampling, learn inter-dimensional correlations and multimodality, and optimize exact likelihoods.
  • Conditional random fields model pixel correlations but require choosing which pixel pairs interact, while adversarial methods can be unstable and difficult to evaluate quantitatively.
  • Conditional normalizing flows condition the prior and invertible mapping on x, and are evaluated on super-resolution and vessel segmentation against factored baselines.

2 BACKGROUND

The background introduces normalizing flows as invertible likelihood models and explains dequantization for discrete data. It also describes why factorized likelihoods inadequately represent structured prediction outputs with correlated dimensions.

  • Normalizing Flows: Normalizing flows model complex distributions by applying a change-of-variables formula to an invertible mapping between equal-dimensional spaces.
  • Normalizing Flows: The Jacobian determinant accounts for volume changes, while useful flow transformations must be bijective, tractable, flexible, and efficiently invertible.
  • Invertible Modules: Affine coupling layers split inputs and nonlinearly transform one component conditioned on the other before reconstructing the output.
  • Invertible Modules: Invertible 1 x 1 convolutions mix channel information, squeeze layers reduce spatial resolution, and split priors model hierarchical correlations while reducing computation.
  • Variational Dequantization: Dequantization introduces continuous latent variables so a continuous normalizing flow can model discrete data, with q(v|y) constrained to valid quantization cells.
  • Structured Prediction: Factored models assume conditional independence across output dimensions, producing uncorrelated sampling noise and often blurry, non-crisp modes.

3 METHOD

The method learns conditional likelihoods with CNFs by conditioning both the prior and bijective mappings on x. It also extends variational dequantization to binary variables using half-infinite noise.

  • 3.1 CONDITIONAL NORMALIZING FLOWS: CNFs learn conditional likelihoods by conditioning the prior and invertible mapping on input x.The conditional flow models pY|X(y|x) with a prior pZ|X(z|x) and a mapping bijective in Y and Z.
  • 3.1 CONDITIONAL NORMALIZING FLOWS: Sampling draws z from an x-conditioned base density before applying inverse bijective mappings to produce y.The base density is a diagonal Gaussian whose parameters depend on x, and the inverse flow can model multimodal conditional outputs.
  • 3.1 CONDITIONAL NORMALIZING FLOWS: Training flows labels y backward to z and evaluates prior log-likelihood in z-space rather than using a handcrafted output-space image loss.Flow and prior parameters are optimized with stochastic gradient descent and minibatches.
  • 3.1 CONDITIONAL NORMALIZING FLOWS: Conditioning enters the prior, split priors, and affine coupling modules through x-dependent parameters and networks.The implementation transforms x into a representation h and concatenates it with relevant flow inputs.
  • 3.2 VARIATIONAL DEQUANTIZATION FOR BINARY RANDOM VARIABLES: The binary extension addresses support mismatch by dequantizing with half-infinite noise.For binary y, the method uses a dequantized continuous representation and sign-dependent noise; softplus keeps neural-network samples positive.

4 RELATED WORK

Related work includes flow-based generative models, conditional flows for decoder and Bayesian prediction settings, and adversarial image-density methods. These approaches differ in conditioning context, training characteristics, sampling, and likelihood evaluation.

  • 4 RELATED WORK: Flow-based generative models are typically trained in data space, with some designed for fast inversion and post-training sampling.Normalizing flows were introduced for flexible variational posteriors, while later generative flows commonly operate directly on data.
  • 4 RELATED WORK: Prior conditional-flow applications include VAE decoders, Bayesian prediction, and semi-supervised learning from unlabeled data.The cited approaches condition flows in different modeling frameworks and learning settings.
  • 4 RELATED WORK: Adversarial methods can generate high-fidelity images but are complicated to train and make likelihood assessment difficult.These properties complicate judging whether models overfit or generalize.

5 EXPERIMENTS

Experiments evaluate CNFs on super-resolution and vessel segmentation against factorized and state-of-the-art baselines using likelihood and task-specific metrics. CNFs improve likelihood and model richer image detail, while achieving competitive conventional metrics across both tasks.

  • 5.1 SINGLE IMAGE SUPER RESOLUTION: The super-resolution CNF outperforms the factorized baseline in log2-likelihood on ImageNet32 and ImageNet64.The comparison is reported in bits per dimension in Table 1.
  • 5.1 SINGLE IMAGE SUPER RESOLUTION: CNF super-resolution samples model more high-frequency detail, including grass and hair, than the factorized baseline mode.The CNF samples use base-distribution temperature τ = 0.8 for best perceptual quality.
  • 5.1.1 EVALUATION: The CNF performs competitively with state-of-the-art super-resolution methods by optimizing likelihood without hyperparameter tuning or compositional loss weighting.The factorized baseline achieves the best SSIM, while the CNF beats it on PSNR; the authors caution that metrics can be misleading.
  • 5.1.1 EVALUATION: For conditional samples, increasing temperature increases high-level detail, while τ = 0.5 balances noise smoothing and detail better than the factorized baseline.The factorized likelihood model produces substantial color noise, whereas the CNF does not show this problem.
  • 5.2 VESSEL SEGMENTATION: On DRIVE vessel segmentation, binary-dequantized CNFs achieve the best bits per dimension with F-scores comparable to DRIU.The CNF does not require ImageNet pretraining, and its mean of 100 samples is used for precision-recall evaluation.

6 CONCLUSION

The paper proposes conditional normalizing flows for learning conditional likelihoods and generalizes variational dequantization to binary variables. Experiments show competitive performance in super-resolution and binary image segmentation.

  • CNFs learn likelihoods of conditional distributions, framing supervised prediction tasks probabilistically.
  • Variational dequantization is generalized to binary random variables for binary segmentation problems.
  • CNFs achieve competitive performance against competing methods in super-resolution and binary image segmentation.

A ARCHITECTURES

The CNF architecture uses conditional coupling layers to create an invertible mapping between latent and output representations, with configurations specified for super-resolution and vessel segmentation.

  • Conditioned on x, the coupling layer builds an invertible mapping between partitioned latent representations z and outputs y.
  • The conditioning input embedding is concatenated to z1 and processed to produce affine transformation parameters applied to z0.
  • The coupling operation is invertible in z and y but not in x.
  • The network follows a multilevel, multisubflow architecture and is optimized with Adam for 200000 iterations.
  • Tables specify separate CNF, coupling-layer, and conditioning-network configurations for the super-resolution and DRIVE segmentation tasks.

B CONDITIONAL IMAGE GENERATION

The paper presents ImageNet64 super-resolution samples from a CNF at temperatures τ = 0, 0.5, and 0.8, illustrating generation under different sampling temperatures.

  • ImageNet64 super-resolution samples are provided at different temperatures τ.
  • Figure 7 shows ImageNet64 super-resolution results sampled at τ = 0.
  • Figure 8 shows ImageNet64 super-resolution results sampled at τ = 0.5.
  • Figure 9 shows ImageNet64 super-resolution results sampled at τ = 0.8.
Loading 1912.00042v2…