PaperlayerView exact revision on arXiv

Source-linked AI summary

Hamiltonian Generative Networks

Peter Toth, Danilo Jimenez Rezende, Andrew Jaegle, Sébastien Racanière, Aleksandar Botev, Irina Higgins

arXiv:1909.13789v2cs.LGstat.ML

TL;DR

Machine learning needs predictive dynamics models, but standard approaches do not provide Hamiltonian properties and learning them from high-dimensional observations raises state-inference and Hamiltonian-learning challenges. The paper introduces HGN to infer abstract states from pixels and learn Hamiltonian rollouts, then extends it into NHF for density modelling. HGN reliably learns dynamics across four simulated systems and supports reversible, controllable rollouts, while NHF provides expressive Hamiltonian flow-based density modelling.

  • Problem

    The paper asks how to learn a system’s Hamiltonian and infer its abstract phase space from the high-dimensional observations typically available to machine learning.

  • Method

    HGN uses a generative model to infer abstract state from pixels and unrolls a learned Hamiltonian according to Hamiltonian equations; NHF chains Hamiltonian transformations for density modelling.

  • Results

    HGN reliably learned Hamiltonian dynamics from noisy pixel observations on four simulated physical systems, while NHF produced expressive density modelling with Hamiltonian properties including volume preservation.

  • Takeaways & Limitations

    The results demonstrate a practical use of Hamiltonian formalism for generative modelling with smooth, conserved, and reversible dynamics and controllable rollout speed.

  • Takeaways & Limitations

    The observation model is presented for high-dimensional noisy visual observations, and the paper’s evaluation uses four simulated physical systems.

Abstract

from arXiv · show

The Hamiltonian formalism plays a central role in classical and quantum physics. Hamiltonians are the main tool for modelling the continuous time evolution of systems with conserved quantities, and they come equipped with many useful properties, like time reversibility and smooth interpolation in time. These properties are important for many machine learning problems - from sequence prediction to reinforcement learning and density modelling - but are not typically provided out of the box by standard tools such as recurrent neural networks. In this paper, we introduce the Hamiltonian Generative Network (HGN), the first approach capable of consistently learning Hamiltonian dynamics from high-dimensional observations (such as images) without restrictive domain assumptions. Once trained, we can use HGN to sample new trajectories, perform rollouts both forward and backward in time and even speed up or slow down the learned dynamics. We demonstrate how a simple modification of the network architecture turns HGN into a powerful normalising flow model, called Neural Hamiltonian Flow (NHF), that uses Hamiltonian dynamics to model expressive densities. We hope that our work serves as a first practical demonstration of the value that the Hamiltonian formalism can bring to deep learning.

1 INTRODUCTION

The paper motivates adapting Hamiltonian dynamics to machine learning because their smoothness, conservation, and reversibility could benefit predictive models. HGN addresses learning Hamiltonians and inferring abstract phase spaces directly from high-dimensional observations.

  • 1 INTRODUCTION: Hamiltonian dynamics predict future behavior from current phase-space states while providing smooth, conserved, and fully reversible time evolution.These properties are presented as useful for machine-learning systems, including agents reasoning about actions and effects.
  • 1 INTRODUCTION: Machine learning must address how to learn a system’s Hamiltonian and infer its abstract phase space from high-dimensional observations.The paper notes that inferred states may include information without a direct physical role.
  • 1 INTRODUCTION: HGN infers an abstract state from pixels and unrolls the learned Hamiltonian according to Hamiltonian equations without restrictive domain assumptions.This is the paper’s proposed answer to both central adaptation questions.
  • 1 INTRODUCTION: HGN learns Hamiltonian dynamics from noisy pixel observations across pendulum, mass-spring, two-body, and three-body systems.The introduction reports reliable learning on four simulated physical systems.
  • 1 INTRODUCTION: A small architectural modification produces Neural Hamiltonian Flow, a normalising-flow model that uses Hamiltonian dynamics for expressive density modelling.The model is described as inheriting Hamiltonian properties including volume preservation and offering computational benefits over standard flow-based models.

2 RELATED WORK

The related-work discussion contrasts HGN with methods that discretize dynamics or require canonical state information. HGN instead targets image observations without assuming phase-space dimensionality or momentum coordinates.

  • 2 RELATED WORK: Discrete-time dynamics models can accumulate approximation errors during rollouts, whereas HGN models continuous dynamics directly with Hamiltonian differential equations.The paper associates direct continuous modelling with slower divergence during longer rollouts.
  • 2 RELATED WORK: HNN learns a differentiable Hamiltonian H(q,p) by matching its gradients to target time derivatives in a supervised setting.Its formulation maps position and momentum to a scalar interpreted as the system Hamiltonian.
  • 2 RELATED WORK: HNN is most directly applicable when the true canonical state space and its time derivatives are known, and most experiments used ground-truth states rather than pixels.Its pixel-observation experiment required an encoder and a phase-space dimensionality matching the ground truth.
  • 2 RELATED WORK: The pixel-based HNN experiment assumed momentum equaled velocity, an assumption appropriate for the simple pendulum but not generally valid.The paper states that HGN makes no assumptions about learned phase-space dimensionality or momentum-coordinate form.

3 METHODS

The paper models Hamiltonian dynamics from high-dimensional image sequences by inferring a latent phase space, learning its Hamiltonian, and decoding simulated states. It extends this architecture into Neural Hamiltonian Flow for density estimation using invertible, volume-preserving Hamiltonian transformations.

  • Hamiltonian Generative Network: HGN infers an abstract phase-space state from noisy observations and learns Hamiltonian dynamics without assuming the phase-space dimensionality or momentum coordinates.The model treats observations as generated from generalized position while inferring momentum from image sequences.
  • Hamiltonian Generative Network: The HGN architecture combines an inference network, Hamiltonian network, and decoder network to map image sequences into latent states and reconstruct observations.The inference network produces a posterior over the initial state, the Hamiltonian network maps states to scalar energies, and the decoder reconstructs pixels from position coordinates.
  • Hamiltonian Generative Network: Hamiltonian rollouts use the learned equations of motion, with leapfrog integration used in practice for better theoretical properties and performance than schematic Euler updates.The training objective combines frame reconstruction terms with a KL term, while rollouts are generated through Hamiltonian equations in the learned phase space.
  • Neural Hamiltonian Flow: NHF modifies HGN into a density model by transforming samples from a simple prior through chained Hamiltonians that are invertible and volume preserving.Unlike HGN, NHF can use a different Hamiltonian at each flow step, and its inverse reverses transformation order while replacing dt with −dt.
  • Neural Hamiltonian Flow: Normalising flows transform simple priors into expressive densities, while NHF uses Hamiltonian dynamics to provide this transformation without requiring the standard Jacobian-trace computation.The density model uses a variational objective with a variational momentum density because the relevant integral is intractable.

4 RESULTS

HGN learned dynamics across four simulated physical systems from image observations, outperforming HNN and supporting reversible, variable-speed rollouts. NHF extended the approach to expressive density modelling with performance comparable to RNVP.

  • Physical-system dynamics: HGN and its modifications learned well on mass-spring, pendulum, two-body, and three-body datasets.Each trajectory contained a 30-step rollout generated from ground-truth Hamiltonian dynamics with added Gaussian noise.
  • Physical-system dynamics: HGN achieved an order-of-magnitude lower average pixel MSE than both HNN versions on train and test rollouts.Table 1 reports average pixel MSE over 30-step unrolls, with values multiplied by 1e+4.
  • Physical-system dynamics: The full HGN produced diverse plausible samples and allowed rollouts to be reversed, sped up, or slowed down by changing the integrator timestep.The three-body example includes forward, backward, double-speed, and half-speed rollouts; HNN instead reconstructed an average image.
  • Physical-system dynamics: The deterministic HGN reconstructed more accurately but could not sample, while Euler integration lacked the leapfrog integrator’s energy-conservation and reversibility properties.Euler integration increased the variance of the learned Hamiltonian by an order of magnitude during sequence rollouts.
  • Expressive density modelling: NHF transformed a simple prior into multimodal densities, with learned potential-energy minima and attractors centered at data modes.The Gaussian-mixture comparison found NHF and RNVP comparable in performance, while NHF was more computationally efficient.

5 CONCLUSIONS

The paper presents HGN as a practical method for learning Hamiltonian dynamics from pixels and demonstrates its extension to computationally efficient density modelling. Its results motivate broader use of Hamiltonian principles in machine learning.

  • Conclusions: HGN reliably learned Hamiltonian dynamics from pixel observations and outperformed the relevant baseline by a large margin.The evaluation covered four classical physical systems.
  • Conclusions: The learned Hamiltonian’s smooth, conserved, and reversible structure may support applications in reinforcement learning, representation learning, and generative modelling.The paper describes NHF as an initial demonstration of Hamiltonian dynamics for expressive, computationally efficient density modelling.

A.1 HAMILTONIAN GENERATIVE NETWORK

HGN combines an encoder, Hamiltonian transition network, and decoder to infer a latent phase-space state from image sequences and reconstruct observations from latent position.

  • Architecture: HGN consists of an encoder, a Hamiltonian transition network, and a decoder.The encoder processes sequences of raw training images before Hamiltonian transitions and decoding.
  • Latent state: The encoder represents each latent spatial location as concatenated abstract position q and momentum p variables.The latent representation is downsized to 4×4, with position and momentum each having dimension 16.
  • Hamiltonian transition: The Hamiltonian transition network applies a symplectic-integrator update with discrete timestep dt=0.125.The Hamiltonian is represented as a function from abstract position and momentum space to a scalar at each timestep.
  • Decoding: At each timestep, the decoder uses only latent position qt to reconstruct an output image with the input shape.The output image has dimensions 32×32×3.
  • Training: Training uses Adam with learning rate 1.5e-4, fixed decoder variance 1, and a Lagrange multiplier on the KL term.With decoder variance fixed to 1, the reconstruction objective becomes a scaled L2 loss.

A.2 NEURAL HAMILTONIAN FLOW

NHF models Hamiltonian flows with separable kinetic and potential energies, using soft-plus MLPs for the energy terms and RNVP as a comparison baseline.

  • Hamiltonian parameterization: NHF parameterizes the Hamiltonian as H(q,p) = K(p) + V(q), using soft-plus MLPs for kinetic and potential energy.The MLP layer sizes are [d,128,128,1], where d is the data dimension; soft-plus supports the required second-order derivatives.
  • Baseline: The density experiments compare NHF with RNVP models using alternating masks and two or three affine-coupling layers.The RNVP coupling networks use two-layer ReLU MLPs matching those used in the leapfrog components.

A.3 HAMILTONIAN NEURAL NETWORK

PixelHNN adapts Hamiltonian Neural Networks to image observations by inferring a low-dimensional position–momentum embedding, while requiring assumptions about phase-space dimensionality and momentum.

  • Training objective: The HNN component learns a differentiable Hamiltonian from the inferred embedding, while the original HNN generally uses ground-truth states and derivatives.The comparison therefore uses PixelHNN for image-based experiments.
  • Architecture: The latent dimensionality is chosen to match the known ground-truth phase-space dimensionality, which is not always available.The pendulum example uses a two-dimensional embedding.
  • Architecture: PixelHNN encodes a concatenated pair of images into latent position and momentum variables before applying an HNN.Two images make momentum observable, and the embedding is used instead of the true system state.
  • Training objective: PixelHNN adds an autoencoding mean-squared-error loss so the embedding reflects image content and avoids a trivial solution.The reconstruction loss compares input pixels with the autoencoder output.
  • Experimental setup: The reimplementation uses MLP encoder, decoder, and HNN subnetworks, while a convolutional variant controls for architectural differences with HGN.The convolutional comparison adds linear mappings between convolutional latent representations and the HNN’s vector-valued latent.
  • Experimental setup: Training uses minibatch Adam with batch size 64, learning rate 1e-3, and approximately 15000 steps, followed by RK4 rollouts.This setup makes PixelHNN more comparable to HGN than the original full-batch training procedure.

A.4 DATASETS

The experiments use rendered image sequences generated from exact simulated Hamiltonian trajectories for four simple physical systems.

  • Dataset generation: Datasets simulate exact Hamiltonian dynamics with a default scipy initial-value problem solver before rendering trajectories as image sequences.The datasets are generated similarly to Greydanus et al. for comparative purposes.
  • Visual representations: Mass-spring and pendulum scenes render the moving mass or bob as a circle while hiding the spring, rod, and pivot.These renderings provide image observations of the underlying trajectories without exposing all physical components.
  • Visual representations: Two- and three-body scenes render each point mass as a differently colored circle, with smoothed rather than hard edges.The smoothing is visible in Fig. 7.

A.5 CONVERGENCE OF HGN AND HNN

HGN and HNN convergence are evaluated on four datasets using fixed training budgets, with reconstruction error tracked during training.

  • Training protocol: Both HGN and HNN are trained for 15000 iterations, using batch sizes 16 and 64 respectively.Given the dataset sizes, this corresponds to approximately 5 HGN epochs and 19 HNN epochs.
  • Convergence measurement: The training runs take around 16 hours, and Figures 11–12 plot convergence rates for HGN and HNN across the four datasets.The figures use average pixel-reconstruction MSE as the tracked quantity.
  • Convergence measurement: Figure 11 presents average pixel-reconstruction MSE during leapfrog-HGN training against training iteration number.The horizontal axis indicates training iteration number in units of 1e+2.

A.6 INTEGRATORS

The paper compares Euler, RK4, and leapfrog integration for Hamiltonian rollouts, emphasizing the long-term stability benefits of symplectic leapfrog updates.

  • Integration methods: HGN estimates future states by numerically integrating inferred position and momentum with Euler, Runge-Kutta, or leapfrog methods.These are the three integration methods explored throughout the paper.
  • Euler integration: Euler extrapolates using only the current derivative, so its errors accumulate rapidly over longer periods or repeated applications.This motivates integrators designed to remain stable over more steps and longer integration times.
  • Runge–Kutta integration: RK4 evaluates derivatives at four points within each interval, producing more stable short-period estimates than Euler.Its four intermediate evaluations better reflect function behavior across the interval.
  • Long-term stability: Neither Euler nor RK4 is guaranteed to preserve system energy indefinitely, and both can drift from true dynamics on relevant simulation timescales.RK4 may be stable over short periods but lacks indefinite stability guarantees.
  • Leapfrog integration: Leapfrog interleaves position and momentum updates for Hamiltonians of the form H = V(q)+T(p).The updates resemble variables leapfrogging over one another at alternating time points.
  • Leapfrog integration: As a symplectic integrator, leapfrog preserves the special Hamiltonian form and is more stable and accurate over long rollouts than Euler or RK4.The paper illustrates this with a harmonic oscillator, where leapfrog conserves phase-space volume while Euler’s volume increases each step.
  • Hamiltonian Flow: The Hamiltonian Flow density model treats qT as observable and pT as latent, requiring marginalization through a variational lower bound.This connects Hamiltonian integration to the paper’s normalising-flow formulation.
Loading 1909.13789v2…