Source-linked AI summary
Learning Stable Deep Dynamics Models
Gaurav Manek, J. Zico Kolter
TL;DR
Generic neural networks make it difficult to guarantee stability when learning dynamical systems. This paper jointly learns dynamics and a Lyapunov function with stability built into the model, demonstrating stable physical-system modeling and end-to-end video-texture dynamics.
Problem
Generic neural-network dynamics models provide little guarantee about learned-system behavior, particularly stability over state space.
Method
The approach jointly learns neural-network dynamics and a convex positive-definite Lyapunov function, directly constraining the dynamics to be stable everywhere.
Results
The models learn physical dynamics such as n-link pendulums and integrate with VAEs to learn complex latent-space video textures end to end.
Takeaways & Limitations
Stable neural-network systems can serve as reusable primitives for modeling physical and complex latent-space dynamics within larger deep architectures.
Takeaways & Limitations
The method requires a Lyapunov function without local optima away from the equilibrium, and the paper focuses on autonomous uncontrolled systems.
Abstract
from arXiv · showhide
Deep networks are commonly used to model dynamical systems, predicting how the state of a system will evolve over time (either autonomously or in response to control inputs). Despite the predictive power of these systems, it has been difficult to make formal claims about the basic properties of the learned systems. In this paper, we propose an approach for learning dynamical systems that are guaranteed to be stable over the entire state space. The approach works by jointly learning a dynamics model and Lyapunov function that guarantees non-expansiveness of the dynamics under the learned Lyapunov function. We show that such learning systems are able to model simple dynamical systems and can be combined with additional deep generative models to learn complex dynamics, such as video textures, in a fully end-to-end fashion.
1 Introduction
The paper learns neural-network dynamical systems that are provably stable across the entire state space by jointly learning dynamics and a Lyapunov function. It demonstrates the approach on physical models and complex video textures using a VAE.
- Motivation: Neural networks model continuous-time dynamics, including systems with control inputs, for applications such as reinforcement learning, control, and forecasting.The dynamics specify a state derivative through f: R^n → R^n, or f(x(t), u(t)) with control inputs.
- Method: The proposed approach jointly learns system dynamics and a Lyapunov function to guarantee stability over the entire state space.Stability is imposed as a hard constraint built directly into the dynamics model rather than enforced through a loss function.
- Experiments: The method learns dynamics for physical systems such as n-link pendulums and substantially improves over generic networks.The passage reports a substantial improvement but provides no numerical value.
- Experiments: The dynamics model integrates with a variational auto-encoder to learn complex-output dynamics such as dynamic video textures.This demonstrates an end-to-end combination of the stable dynamics model with a larger generative network system.
2 Background and related work
This section reviews Lyapunov-based stability analysis for autonomous dynamical systems and explains why jointly learning stable dynamics and Lyapunov functions is challenging. It motivates the proposed approach, which learns provably stable systems by enforcing stability according to a learned Lyapunov function rather than finding one for a fixed system.
- Stability of dynamical systems: Lyapunov theory connects stability to descent of a continuously differentiable positive definite function.The relevant descent condition must hold over the entire state space for global stability or within a region for local stability.
- Stability of linear systems: For linear systems, stability is characterized by strictly negative real eigenvalue components or an equivalent positive definite quadratic Lyapunov function.Linear matrix inequalities provide a framework for enforcing these stability conditions and have also been applied to nonlinear systems through constrained linearizations.
- Stability of linear systems: Jointly learning a stable linear dynamics matrix and its Lyapunov function is non-convex, despite convexity in each variable separately.This motivates alternative procedures such as alternating minimization and approximate projection heuristics.
- Stability of non-linear systems: For nonlinear systems, finding a Lyapunov function is generally difficult and often relies on problem-specific reasoning or certification methods such as sum-of-squares.The difficulty is especially pronounced when certifying stability for a known nonlinear dynamics function.
- Stability of non-linear systems: The proposed approach avoids finding a Lyapunov certificate for a fixed system by enforcing the learned dynamics to behave stably according to a learned Lyapunov function.This enables learning systems with provable stability without solving the generally hard function-finding problem.
- Lyapunov functions in deep learning: Prior deep-learning work learned neural-network Lyapunov functions for control policies and enforced stability through loss penalties, differing from the proposed approach.These methods lie at the intersection of deep learning and Lyapunov analysis but follow a different control-oriented formulation.
3 Joint learning of dynamics and Lyapunov functions
The method jointly learns a dynamics model and Lyapunov function, projecting the dynamics to satisfy Lyapunov-based stability constraints everywhere. Carefully designed Lyapunov networks yield globally exponentially stable dynamics while allowing optional invertible input warping to reduce restrictions from strong convexity.
- Joint learning: The approach jointly learns a dynamics model and Lyapunov function, constraining the dynamics to be stable everywhere according to the learned Lyapunov function.Rather than separately verifying a learned dynamics model, the method incorporates stability into the learned dynamics.
- Joint learning: The dynamics are obtained by projecting a nominal model onto a Lyapunov-defined halfspace, with gradients computed by automatic differentiation.The projection is orthogonal, and its analytical halfspace form can be incorporated into a trainable network.
- Lyapunov function design: The Lyapunov function is designed to be positive definite, have no local optima except x = 0, and be continuously differentiable.An ICNN enforces convexity, while positive-definite terms and smoothed activations provide the remaining properties.
- Lyapunov function design: An optional continuously differentiable invertible input transformation can precede the ICNN, avoiding the overly restrictive requirement of a strongly convex Lyapunov function.Invertibility preserves the absence of local optima in the composed function by mapping convex sublevel sets to contiguous regions.
- Stability guarantee: The resulting dynamics are globally exponentially stable to x = 0 for any bounded-weight networks defining the nominal dynamics and Lyapunov functions.The construction also ensures that the dynamics are defined and bounded everywhere.
4 Empirical results
The method produces stable dynamics before training, accurately models damped pendulums over long horizons, and generates realistic video textures with stable trajectories. Across these experiments, the learned dynamics capture contraction and avoid the divergence seen with unconstrained models.
- Random networks: Randomly initialized networks already produce stable dynamics with interesting behaviors because stability is built into the method.The example uses a 2-100-100-2 dynamics network and a 2-100-100-1 ICNN Lyapunov function.
- n-link pendulum: For a simple damped pendulum, the learned vector field and trajectories accurately predict system motion over long time periods.The experiment compares simulated and learned dynamics while visualizing the learned Lyapunov function.
- n-link pendulum: As pendulum size and simulation horizon vary, the stable model captures physical contraction, decreasing error toward the simulation’s end while the simple model’s error increases.The comparison includes 8-link pendulums and average cumulative error across different numbers of links.
- Video textures: The method combines a VAE with stable latent-space dynamics to generate video textures by encoding frames, evolving the latent state, and decoding trajectories into frames.Training minimizes both the VAE objective and next-frame reconstruction loss.
- Video textures: Unconstrained dynamics quickly diverge into a static image, whereas the stable model generates different trajectories that continue producing realistic images over long horizons.The stable video trajectories are learned in a 320-dimensional latent space and projected to two dimensions for display.
5 Conclusion
The paper proposes jointly learning neural-network dynamics and a convex positive definite Lyapunov function to guarantee stability throughout the state space. These models also integrate with VAEs for end-to-end learning of complex latent-space dynamics.
- 5 Conclusion: The method learns stable non-linear dynamical systems defined by neural network architectures.It jointly learns the dynamics and a convex positive definite Lyapunov function.
- 5 Conclusion: The learned dynamics are constrained to remain stable everywhere in the state space.The stability constraint is defined according to the jointly learned Lyapunov function.
- 5 Conclusion: The models integrate with VAEs to learn complex latent-space dynamics in a fully end-to-end manner.The paper identifies VAEs as an example of another deep architecture with which the models can be integrated.