On the Anatomy of MCMC-Based Maximum Likelihood Learning of Energy-Based Models

Erik Nijkamp; Mitch Hill; Tian Han; Song-Chun Zhu; Ying Nian Wu

Source-linked AI summary

On the Anatomy of MCMC-Based Maximum Likelihood Learning of Energy-Based Models

Erik Nijkamp, Mitch Hill, Tian Han, Song-Chun Zhu, Ying Nian Wu

arXiv:1903.12370v4Mar 29, 2019stat.MLcs.CVcs.LG

TL;DR

The paper asks how MCMC implementation affects maximum-likelihood learning of ConvNet energy-based models and whether learned energies represent realistic image densities. It analyzes two learning axes, develops noise-initialized and properly tuned Langevin procedures, and reports realistic short-run synthesis together with the first realistic steady-state samples under its scope.

Problem
Prior ConvNet energy models had realistic short-run samples but long-run samples that failed to reflect the training-data density.
Method
The paper analyzes energy-difference and MCMC-convergence axes and uses noise-initialized MCMC plus correctly tuned Langevin sampling to train ConvNet potentials.
Results
The method produces realistic, diverse samples from noise and ConvNet potentials with realistic long-run and steady-state samples, unlike prior implementations.
Takeaways & Limitations
High-quality synthesis does not require MCMC convergence, but realistic steady-state learning requires informative initialization and precise Langevin-noise tuning.
Takeaways & Limitations
Informative MCMC initialization can limit the final model’s ability to generate new and diverse images from noise.

Abstract

from arXiv · show

This study investigates the effects of Markov chain Monte Carlo (MCMC) sampling in unsupervised Maximum Likelihood (ML) learning. Our attention is restricted to the family of unnormalized probability densities for which the negative log density (or energy function) is a ConvNet. We find that many of the techniques used to stabilize training in previous studies are not necessary. ML learning with a ConvNet potential requires only a few hyper-parameters and no regularization. Using this minimal framework, we identify a variety of ML learning outcomes that depend solely on the implementation of MCMC sampling. On one hand, we show that it is easy to train an energy-based model which can sample realistic images with short-run Langevin. ML can be effective and stable even when MCMC samples have much higher energy than true steady-state samples throughout training. Based on this insight, we introduce an ML method with purely noise-initialized MCMC, high-quality short-run synthesis, and the same budget as ML with informative MCMC initialization such as CD or PCD. Unlike previous models, our energy model can obtain realistic high-diversity samples from a noise signal after training. On the other hand, ConvNet potentials learned with non-convergent MCMC do not have a valid steady-state and cannot be considered approximate unnormalized densities of the training data because long-run MCMC samples differ greatly from observed images. We show that it is much harder to train a ConvNet potential to learn a steady-state over realistic images. To our knowledge, long-run MCMC samples of all previous models lose the realism of short-run samples. With correct tuning of Langevin noise, we train the first ConvNet potentials for which long-run and steady-state MCMC samples are realistic images.

1 Introduction

This paper studies MCMC-based maximum-likelihood learning of ConvNet energy-based models, focusing on the gap between realistic short-run synthesis and realistic long-run sampling. It identifies two learning axes and introduces approaches addressing noise-initialized synthesis and steady-state realism.

1 Introduction: Unsupervised energy-based modeling is used for high-dimensional unlabeled images, where supervised deep-learning tools cannot directly apply.
1 Introduction: Prior ConvNet energy models produce realistic short-run samples, but their long-run and steady-state samples become unrealistic and lower-energy than observed data.
1 Introduction: The paper characterizes each ML update along energy difference between data and synthesized samples and MCMC convergence toward steady-state.
1 Introduction: The authors report that convergence is unnecessary for high-quality synthesis, while realistic steady-state sampling requires proper MCMC implementation.
1 Introduction: Noise-initialized MCMC enables high-diversity realistic synthesis after training, unlike the informative-initialization approaches used in prior work.
1 Introduction: The paper also maps energy landscapes through diffusion to support unsupervised cluster discovery.

2 Learning Energy-Based Models

Maximum Likelihood learning fits a ConvNet energy-based model by contrasting data samples with MCMC-generated model samples. The sampling initialization and chain behavior determine whether the learned model supports realistic synthesis and a realistic steady-state.

2 Learning Energy-Based Models: An energy-based model uses a Gibbs-Boltzmann density over signals, with an intractable normalization constant Z(θ).The energy potential supplies sufficient information for MCMC sampling, so Z(θ) need not be used explicitly.
2 Learning Energy-Based Models: ML learning seeks parameters θ such that the model distribution pθ approximates the data distribution q, using KL divergence as a closeness measure.The objective includes the log partition function and the expected energy under q.
2 Learning Energy-Based Models: Positive samples come from the data distribution, while negative samples come from the current learned distribution pθ and approximate the intractable gradient term.In practice, positive samples are training images and negative samples are generated by MCMC.
2 Learning Energy-Based Models: Langevin dynamics is used to obtain negative samples because Gibbs sampling is computationally infeasible for ConvNet energies at standard image sizes.For sufficiently small ε, momentum and Metropolis-Hastings updates are commonly omitted in practice.
2 Learning Energy-Based Models: Langevin chains have high autocorrelation and difficulty mixing between separate modes, making initialization important for approximate steady-state sampling.Informative initialization can converge faster than noise initialization, while noise initialization is a non-informative alternative.
2 Learning Energy-Based Models: Informative initialization is not necessary for efficient realistic synthesis, but it is essential for learning a realistic steady-state in ConvNet-potential ML.Data-based initialization includes CD, and persistent initialization includes PCD; informative initialization can nevertheless limit post-training diversity from noise.

3 Two Axes of ML Learning

MCMC-based ML updates are governed by two axes: whether the energy difference drives expansion or contraction, and whether finite-step MCMC converges. The first axis controls learning stability and synthesis, while the second determines whether steady-state samples are realistic.

Energy difference: A large |d_st| indicates that learning, sampling, or both have not converged; perfect learning with exact MCMC convergence gives d_st(θ)=0.d_st compares the average energies of positive data samples and negative MCMC samples.
Two axes: Each update is classified by the sign of d_st(θ_t), indicating expansion or contraction, and by whether s_t approximates p_θt.These axes separate energy-update behavior from MCMC convergence behavior.
Two axes: Only expansion or contraction governs learning stability and synthesis, whereas MCMC convergence determines the realism of final steady-state samples.Expansion–contraction oscillation can occur with either convergent or non-convergent chains.
MCMC convergence: Informative initialization is unnecessary for efficient realistic short-run synthesis but essential for learning a realistic steady-state.The paper contrasts the ease of short-run synthesis with the difficulty of obtaining realistic steady-state behavior.
MCMC convergence: Low noise can make noise-initialized paths nearly linear and prevent mixing, while sufficiently large noise can enable convergence and realistic steady-state learning.With low noise, the model may learn only the burn-in path or collapse toward a constant function; with sufficient noise, gradients can balance the noise magnitude.
Training setup: The method requires no auxiliary regularization or normalization techniques used in previous studies.The authors report that prior stabilizing techniques are unnecessary in their minimal ML framework.

4 Experiments

Experiments compare non-convergent and convergent ML across toy distributions and image synthesis, showing that MCMC initialization, Langevin noise, and convergence determine distinct outcomes.

4.1 Low-Dimensional Toy Experiments: Non-convergent models trained with noise initialization and 100 Langevin steps produced short-run samples reflecting ground-truth densities but sharply concentrated learned densities.The non-convergent setting used ε = 0.01, described as too low for the data temperature.
4.2 Synthesis from Noise with Non-Convergent ML Learning: Noise-initialized ML generated high-fidelity, diverse images from scratch using the same Langevin budget as informative initialization.The method's results are shown for noise-initialized non-convergent ML in Figures 7 and 8.
4.2 Synthesis from Noise with Non-Convergent ML Learning: Figure 8 compares negative and steady-state samples for noise-initialized non-convergent ML, vanilla-ConvNet convergent ML, and non-local-network convergent ML.The compared methods use 100 steps on the left and 500 steps in the center and right panels.
4.3 Convergent ML Learning: Correct Langevin noise can keep MCMC samples in the steady-state energy spectrum, but noise initialization required approximately 20,000 steps for convergent ML.Convergent learning requires approximate MCMC convergence after a burn-in period.
4.3 Convergent ML Learning: Persistent initialization with 500 sampling steps trained convergent models for both a vanilla ConvNet and a network with non-local layers.The procedure initialized 10,000 persistent images from noise and updated 100 images per batch.
4.4 Energy Landscape: The basin visualization groups metastable samples by reachable travel in a magnetized energy landscape and displays basin members, minima, and merging barriers.This mapping examines the structure of a convergent energy function on Oxford Flowers 102.

5 Conclusion and Future Work

The paper identifies two axes governing MCMC-based ML and uses them to obtain realistic synthesis from noise and realistic steady-state samples. These findings expose defects in prior implementations and suggest directions for more complex initialization methods.

Conclusion: The experiments identify two distinct ML axes: the energy difference between positive and negative samples, and MCMC convergence or non-convergence.The paper states that convergence is not needed for high-quality synthesis.
Conclusion: Removing informative initialization yields energy functions capable of high-diversity, realistic synthesis from noise after training.The paper presents these as the first such energy functions in its setting.
Conclusion: The paper introduces the first ConvNet potentials in its setting whose steady-state samples have realistic appearance.It contrasts this capability with previous training implementations, whose realistic image-space MCMC sampling was unobtainable according to the authors.