Continuity-Preserving Convolutional Autoencoders for Learning Continuous Latent Dynamical Models from Images

Aiqing Zhu; Yuting Pan; Qianxiao Li

Source-linked AI summary

Continuity-Preserving Convolutional Autoencoders for Learning Continuous Latent Dynamical Models from Images

Aiqing Zhu, Yuting Pan, Qianxiao Li

arXiv:2502.00754v1Feb 2, 2025cs.LGcs.CV

TL;DR

Discrete image frames can yield discontinuous latent coordinates even when the underlying dynamics are continuous. The paper introduces CpAEs that promote convolution-filter continuity to preserve continuous latent states and learn corresponding latent dynamics, with experiments showing improved predictive accuracy over standard CNN autoencoders while retaining a continuous latent model.

Problem
Image observations are discrete, and standard autoencoders can produce latent representations that do not evolve continuously with the underlying dynamics.
Method
CpAEs combine a mathematical continuity formulation with filter-continuity regularization so latent states preserve the continuity of the underlying dynamics.
Results
CpAEs accurately approximate dynamics and outperform standard CNN autoencoders in predictive accuracy while producing a continuous latent model.
Takeaways & Limitations
The resulting continuous latent model supports scientific applications requiring continuous dynamics, including time reversal and interpolation.
Takeaways & Limitations
Continuity quantification is limited to rigid-body motion in two dimensions, and performance declines for visually complex, noisy tasks such as swing-stick dynamics.

Abstract

from arXiv · show

Continuous dynamical systems are cornerstones of many scientific and engineering disciplines. While machine learning offers powerful tools to model these systems from trajectory data, challenges arise when these trajectories are captured as images, resulting in pixel-level observations that are discrete in nature. Consequently, a naive application of a convolutional autoencoder can result in latent coordinates that are discontinuous in time. To resolve this, we propose continuity-preserving convolutional autoencoders (CpAEs) to learn continuous latent states and their corresponding continuous latent dynamical models from discrete image frames. We present a mathematical formulation for learning dynamics from image frames, which illustrates issues with previous approaches and motivates our methodology based on promoting the continuity of convolution filters, thereby preserving the continuity of the latent states. This approach enables CpAEs to produce latent states that evolve continuously with the underlying dynamics, leading to more accurate latent dynamical models. Extensive experiments across various scenarios demonstrate the effectiveness of CpAEs.

1 INTRODUCTION

Image observations discretize continuous dynamics at the pixel level, making standard autoencoders prone to discontinuous latent representations. The paper introduces CpAEs, which promote filter continuity to learn continuously evolving latent states and dynamics.

Discrete image measurements can misalign with continuous underlying dynamics, while standard autoencoders may produce discontinuous latent states.This undermines learning continuous latent dynamical systems from image sequences.
Continuity-preserving convolutional autoencoders (CpAEs) learn continuously evolving latent states from discrete image frames.
The paper formulates learning continuous dynamics from images, establishes filter Lipschitz continuity as sufficient for continuous latent evolution, and introduces a continuity-promoting regularizer.Experiments across varied scenarios evaluate the proposed method.
Related Work: Autoencoders compress inputs into latent representations and reconstruct them, and this paper adapts them for learning latent dynamical models from images.
Related Work: Image-based dynamical-modeling research incorporates physical, geometric, and symmetry-based inductive biases, while this work prioritizes continuous latent evolution with deterministic autoencoders.Prior approaches have often been tested on relatively simple visual patterns and dynamics.

2 LEARNING LATENT DYNAMICAL MODELS USING AUTOENCODERS

The task is to infer continuous latent dynamics and reconstruct images from discrete frames generated by an unknown system. Its framework combines an encoder, latent dynamical model, and decoder, with continuity of latent trajectories treated as a prerequisite.

The dataset contains discrete image frames sampled from multiple continuous trajectories of an unknown system.
The unknown system is represented through physical states, an image-observation mapping, and a governing function assumed to be Lipschitz continuous and bounded.
The framework jointly targets an encoder producing consistent latent states, a dynamical model predicting latent evolution, and a decoder reconstructing pixel observations.The encoder maps images to latent states, the dynamical model advances them, and the decoder reconstructs images.
Continuous latent trajectories are a prerequisite for identifying a meaningful latent dynamical system with assumed structure.The encoder is designed to ensure latent variables evolve continuously over time.

3 CONTINUITY-PRESERVING AUTOENCODER

The paper formulates continuity for latent states extracted from discrete images and shows why unconstrained CNN encoders can break it. It then links filter regularity to latent continuity and motivates continuity-preserving autoencoders.

3.1 MATHEMATICAL FORMULATION: The image pipeline maps system states to particle positions, discretizes occupied regions into binary pixels, and encodes the resulting images into latent states.The observed image is defined as Iδ ◦ S(z), and the encoder operates on this discretized representation.
3.1 MATHEMATICAL FORMULATION: The autoencoder must make encoded states δ-continuous while keeping decoding approximately invertible, ensuring non-trivial latent states that follow the underlying dynamics.Under sufficiently small pixel size, successive latent differences are bounded by the corresponding state differences and time step.
3.1 MATHEMATICAL FORMULATION: Discrete pixelization can hide small state variations, so the extracted image representation cannot satisfy standard continuity even when the underlying dynamics are continuous.The paper introduces δ-continuity as a relaxation suited to discrete pixel observations.
3.3 WHY STANDARD CNN AUTOENCODERS FAIL: Standard CNN filters can fail to preserve continuity: fixed-size filters produce non-δ-continuous outputs, while independently sampled expanding filters fail with probability one.The failure is especially problematic for small objects whose positions overlap only partially between successive observations.
3.4 QUANTIFYING CONTINUITY OF CNN AUTOENCODERS: Theorem 3.1 states that Lipschitz-continuous filter functions in the first few CNN layers are sufficient for latent states to evolve continuously with rigid-body dynamics.This establishes the connection between filter continuity and encoder δ-continuity.

4 EXPERIMENTS

Experiments evaluate CpAEs on circular motion, simulated pendulums, and real-world systems using continuous latent-state learning and predictive metrics. CpAEs preserve latent continuity and achieve accurate dynamics prediction, outperforming standard autoencoders particularly on real-world data.

4 EXPERIMENTS: The benchmark compares Neural ODEs, HNNs, SympNets, hybrid neural state variables, and CpAEs paired with continuous latent dynamics.Neural ODEs use a continuity prior, whereas HNNs and SympNets incorporate Hamiltonian-system structure.
4 EXPERIMENTS: Performance is evaluated with Valid Prediction Time and Valid Prediction Frequency, using thresholded image-prediction errors averaged across test trajectories.VPT uses dataset-specific thresholds, while VPF measures the frequency of trajectories with VPT = 1.
4.1 CONTINUITY OF LATENT STATES: CpAEs learn continuous latent states that enable a Neural ODE to capture circular-motion dynamics, whereas standard and L2-regularized autoencoders fail.The continuity regularizer produces continuous latent evolution and permits accurate image prediction through the learned Neural ODE.
4.2 SIMULATION DATA: CpAEs and hybrid neural-state-variable schemes accurately approximate simulated dynamics, while standard CNN autoencoders have lower predictive accuracy from discontinuous latent states.CpAEs additionally provide a continuous latent model that supports time reversal and interpolation between observed states.
4.3 REAL-WORLD DATA: CpAEs outperform baseline methods on real-world double-pendulum and swing-stick data, including systems where standard autoencoders violate relevant latent constraints.The double-pendulum case is particularly challenging because standard autoencoders struggle to learn variables adhering to Hamiltonian constraints.

5 SUMMARY

The paper introduces CpAEs to impose continuity on latent representations extracted from image frames. Its analysis links filter regularity to latent continuity, while experiments show improved performance over standard autoencoders; the method remains limited in scope and challenging visual settings.

5 SUMMARY: CpAEs impose continuity on latent state representations and outperform standard autoencoders in numerical experiments.The paper formulates learning dynamics from image frames and analyzes discontinuities produced by standard CNN encoders.
5 SUMMARY: Lipschitz-continuous filters provide a sufficient condition for CNN encoder outputs to evolve continuously with the underlying dynamics.The method promotes filter continuity through regularization.
5 SUMMARY: The continuity analysis is limited to rigid-body motion in a two-dimensional plane, while performance declines on visually complex, noisy swing-stick tasks.The paper identifies non-rigid motion and three-dimensional-to-two-dimensional projection as future generalization challenges.

A.1 RIGID MOTION MODELING

The rigid-motion model represents each body state through translation and rotation from a reference placement. Fixed inter-particle distances make this representation applicable to rigid bodies moving in a two-dimensional plane.

A.1 RIGID MOTION MODELING: A rigid body is modeled by combining translation of a reference point with a single orientation angle.The reference particle is typically chosen at the body’s center of mass or centroid.
A.1 RIGID MOTION MODELING: Each particle’s current position is obtained by rotating its reference-relative vector and adding the reference-point translation.The rotation is represented by Φθ, while the full body state is z = (r, θ).
A.1 RIGID MOTION MODELING: Rigid-body motion preserves every inter-particle distance between the reference and current placements.This invariant is the defining constraint used in the model.
A.1 RIGID MOTION MODELING: For multiple objects, the two-dimensional rigid-body dynamics are formed by applying the translation-and-rotation construction to each object.The paper denotes the corresponding multi-object formulation in Eq. (4).

A.2.1 FUNCTION REPRESENTATION OF CNN AUTOENCODERS

The CNN autoencoder is represented as spatial feature-map and filter functions on a two-dimensional grid. Striding changes the evaluation set and downsampling can disrupt continuity, motivating conditions on filters and layer geometry.

A.2.1 FUNCTION REPRESENTATION OF CNN AUTOENCODERS: CNN feature maps and filters are represented as functions on a two-dimensional plane, with each layer evaluated on a stride-dependent set.The filter of layer l is represented by ϕl : R2 × R2 → R.
A.2.1 FUNCTION REPRESENTATION OF CNN AUTOENCODERS: The input image is modeled as a piecewise-constant approximation of the indicator function of the occupied set, partitioned with grid size δ.The representation is zero outside the unit square.
A.2.1 FUNCTION REPRESENTATION OF CNN AUTOENCODERS: A stride of 2 halves the feature-map size and removes every other element from the previous evaluation set.Activation layers do not change the evaluation set.
A.2.1 FUNCTION REPRESENTATION OF CNN AUTOENCODERS: Under the stated assumption and layer-stride conditions, the appendix gives an equivalent theorem governing continuity behavior as δ approaches zero.The theorem introduces a constant C independent of δ and z.

A.2.2 THE CASE OF L∗= 1

The proof handles the L*=1 case under volume-preserving latent transformations whose displacement is bounded by the latent shift.

A.2.2 THE CASE OF L∗= 1: The L*=1 case is concluded after establishing this bound.
A.2.2 THE CASE OF L∗= 1: Under the stated assumptions, rigid body translations and rotations satisfy the required transformation conditions.The image of z is represented as S(z), and Φ_∆ maps S(z) to S(z + ∆) with bounded displacement.
A.2.2 THE CASE OF L∗= 1: The proof derives a bound proportional to ∥∆∥ for the relevant expression.The bound uses the constants c1 and cϕ.

A.2.3 THE CASE OF RIGID BODY TRANSLATIONAL MOTION

This section establishes translation-related identities for convolutional feature maps and uses them to extend the rigid-body argument through the network.

A.2.3 THE CASE OF RIGID BODY TRANSLATIONAL MOTION: Equivalent evaluation sets produce identical convolution outputs, allowing the analysis to replace evaluation sets without changing results.
A.2.3 THE CASE OF RIGID BODY TRANSLATIONAL MOTION: Weight sharing gives convolutional filters translational invariance, yielding the filter-weight property used in the feature-map proof.
A.2.3 THE CASE OF RIGID BODY TRANSLATIONAL MOTION: The feature map obeys a translation-equivariance identity that shifts both the evaluation points and the input coordinate.For K=1, the identity is proved by induction across hidden layers.
A.2.3 THE CASE OF RIGID BODY TRANSLATIONAL MOTION: The argument extends from K=1 to multiple translation components through recursively defined variations and repeated application of the preceding lemmas.The construction uses ∆=(∆1,…,∆K) and concludes with the corresponding corollary.

A.2.4 PROOF OF THEOREM A.1

The theorem proof decomposes latent-state differences into translation and temporal components, while experiments evaluate reconstruction, prediction, and continuity across several systems.

A.2.4 PROOF OF THEOREM A.1: Continuous models support both backward prediction and interpolation by changing the integrator’s dt value or sign.Figure 10 presents backward predictions and interpolations as applications of this flexibility.
A.2.4 PROOF OF THEOREM A.1: CpAEs generate latent states that evolve continuously over time, unlike standard autoencoders and conventional L2 regularization in the reported prediction setting.The latent-state behavior is visualized in Figure 11, while Figure 14 shows the associated image predictions.
A.2.4 PROOF OF THEOREM A.1: All FNN-based methods except HNN accurately predict the two-body trajectory within the evaluated time interval.The comparison is reported for the two-body dataset in Figure 12.
A.2.4 PROOF OF THEOREM A.1: FNN autoencoders fail to completely reconstruct damped and elastic double pendulum images with more complex visual patterns.These reconstruction results motivate focusing on CNN-based autoencoders.
A.2.4 PROOF OF THEOREM A.1: The continuity regularizer balances filter continuity against model complexity, because excessive weighting can underfit while insufficient weighting risks discontinuous latent states.The trade-off is described through λJ and illustrated in Figure 15.