Source-linked AI summary
Deep Koopman Operator with Control for Nonlinear Systems
Haojie Shi, Max Q. -H. Meng
TL;DR
Unknown nonlinear dynamics and nonlinear control inputs make Koopman embedding design and linear-control modeling difficult. The paper jointly learns the embedding and operator, adds an auxiliary control network for nonlinear control terms, and reports improved prediction and control across several nonlinear systems.
Problem
Koopman methods require suitable embeddings and often assume linear control inputs, limiting prediction and control for fully nonlinear systems.
Method
The framework jointly learns the Koopman embedding and operator, encodes nonlinear state-dependent control with an auxiliary network, and applies LQR in the embedding space.
Results
Experiments report better prediction and control than existing Koopman-based and deep-learning approaches across nonlinear dynamic systems.
Takeaways & Limitations
The approach supports linear-control methods for nonlinear systems with highly nonlinear control terms while retaining strong prediction performance.
Takeaways & Limitations
The method does not consider input and state constraints, and recovering real control signals from DKN auxiliary-network outputs lacks a unique inverse mapping.
Abstract
from arXiv · showhide
Recently Koopman operator has become a promising data-driven tool to facilitate real-time control for unknown nonlinear systems. It maps nonlinear systems into equivalent linear systems in embedding space, ready for real-time linear control methods. However, designing an appropriate Koopman embedding function remains a challenging task. Furthermore, most Koopman-based algorithms only consider nonlinear systems with linear control input, resulting in lousy prediction and control performance when the system is fully nonlinear with the control input. In this work, we propose an end-to-end deep learning framework to learn the Koopman embedding function and Koopman Operator together to alleviate such difficulties. We first parameterize the embedding function and Koopman Operator with the neural network and train them end-to-end with the K-steps loss function. Then, an auxiliary control network is augmented to encode the nonlinear state-dependent control term to model the nonlinearity in the control input. This encoded term is considered the new control variable instead to ensure linearity of the modeled system in the embedding system.We next deploy Linear Quadratic Regulator (LQR) on the linear embedding space to derive the optimal control policy and decode the actual control input from the control net. Experimental results demonstrate that our approach outperforms other existing methods, reducing the prediction error by order of magnitude and achieving superior control performance in several nonlinear dynamic systems like damping pendulum, CartPole, and the seven DOF robotic manipulator.
I. INTRODUCTION
The paper addresses unknown nonlinear-system control by learning Koopman embeddings that support linear control while representing nonlinear control inputs. Its framework combines end-to-end operator learning with an auxiliary control network and demonstrates improved prediction and control across several systems.
- Unknown or complex nonlinear dynamics make model-based prediction and state-feedback or optimization-based control difficult.
- Koopman methods map nonlinear dynamics into a linear embedding space, but embedding selection, distorted embedded costs, and linear-input assumptions remain challenges.
- The proposed framework jointly learns the embedding function and Koopman Operator while preserving the original state for consistency between embedded and original cost functions.
- An auxiliary control network encodes state-dependent nonlinear control terms as a new control variable, preserving modeled linear dynamics for linear control methods.
- Experiments report better prediction and control than existing Koopman-based methods across damping pendulum, CartPole, and seven DOF robotic-manipulator environments.
III. BACKGROUND
Koopman Operator theory lifts nonlinear state dynamics into an embedding space where their evolution is represented linearly. Earlier controlled formulations simplify the control term, which can damage prediction and control performance.
- For discrete nonlinear dynamics, the Koopman Operator evolves embedding functions of the state through a linear operator.
- With x∈R^n, u∈R^m, and g:R^(n+m)→R^d, the embedding function lifts the state space and Koopman Operator maps nonlinear dynamics to linear dynamics.
- Previous methods separate the embedding into state and control parts, assuming the state term depends only on state and the control term equals the input.
- This control-term simplification inevitably damages nonlinear-system prediction and can further affect control results.
B. Learning Koopman Operator
The learning framework parameterizes the state embedding with a neural network while retaining the original state, then offers progressively richer representations for nonlinear control terms.
- Data-driven Koopman learning estimates the operator from embedded state-transition data by minimizing mean square prediction loss.
- The state embedding is parameterized by a deep neural network and concatenated with the original state, allowing state recovery and preservation of the control cost form.
- DKUC simplifies the control embedding to the raw input, reducing the evolution model to the standard linear-control form.
- DKAC represents the control embedding in control-affine form and parameterizes its state-dependent factor with an auxiliary control network.
- DKN uses the control network to approximate the control embedding directly as a function of state and input.
B. Feedforward Prediction
Feedforward prediction uses learned embedding and operator parameters to propagate future states over multiple steps. The control representation varies across the proposed algorithm variants.
- The model predicts future K-step states with a feedforward network from the current state while parameterizing Koopman matrices as linear-network matrices A and B.
- The control embedding is set to the input for KPUC, a state-dependent affine form for KPAC, and a direct state-and-input function for KPN.
C. K-steps Loss Function
The framework learns the embedding function and Koopman Operator end-to-end using a K-steps loss for long-horizon prediction, then applies LQR in embedding space with encoded control as the control variable.
- K-steps prediction: The K-steps loss jointly trains the embedding function and Koopman Operator for long-term prediction.It minimizes a weighted sum of prediction errors across K steps rather than only one-step error.
- Control modeling: The auxiliary control network encodes state-dependent nonlinear control, allowing the modeled dynamics to remain linear in embedding space.The encoded term is treated as the new control variable for linear control.
- LQR control: LQR computes the optimal policy in embedding space, after which the actual control is recovered through the control network.The embedded problem becomes an LQR problem with a closed-form solution via the LQR algorithm.
V. EXPERIMENTS
The experiments compare prediction, control, and control-network effectiveness against prior Koopman-based and deep learning-based methods across nonlinear systems.
- Experimental questions: The experiments evaluate prediction performance and sample efficiency against previous Koopman-based and deep learning-based approaches.These comparisons address both predictive accuracy and data requirements.
- Experimental questions: The experiments compare control performance with previous Koopman-based methods using linear control.This evaluates whether the proposed approaches improve nonlinear-system control relative to those baselines.
- Experimental questions: The experiments test whether the control network improves prediction and control by comparing DPAC and DKN with DKUC.The comparison isolates the contribution of the proposed nonlinear-control modeling.
A. Environments
The evaluation uses six nonlinear dynamics simulation environments, including pendulum, vehicle, cart, double-pendulum, and robotic-manipulator tasks.
- Evaluation environments: The study evaluates six environments: DampingPendulum, Pendulum, MountainCarContious, CartPole, DoublePendulum, and Franka.Pendulum, MountainCarContious, and CartPole are modified from OpenAI Gym, while Franka matches the environment in.
- System settings: The DampingPendulum environment models mass, length, damping, and an external horizontal control force.The external force is the control input in this environment.
- System settings: The DoublePendulum dynamics use a mass matrix, velocity-product term, gravity term, and input force u∈R2.These terms define the partial-derivative dynamics used for the environment.
B. K-steps Prediction
The prediction study compares Koopman and deep learning approaches over 30-step horizons, finding stronger long-horizon performance for deep methods and benefits from nonlinear control encoding.
- Compared methods: The study compares KDerivative, KRBF, KDNN, KRNN, and the proposed methods across six nonlinear environments.KDerivative and KRBF use linear regression, while KDNN and KRNN learn nonlinear evolution functions with K-steps loss.
- Prediction results: 30-step results show least-squares Koopman methods perform better initially, while deep learning methods perform better over longer horizons.The comparison uses the mean log10 maximum error over 30 time steps and reports 15th-step errors in Table I.
- Prediction results: DKN performs best in most cases and can outperform KRNN despite KRNN using additional temporal information.This result is reported across the evaluated nonlinear environments.
- Prediction results: DKUC and DKAC reduce prediction error by magnitudes relative to KDerivative and KRBF and match or exceed KDNN and KRNN.DKAC further outperforms DKUC, especially on DampingPendulum.
C. Sample Efficiency of Learning Algorithms
DKUC and DKAC maintain prediction performance across training-sample counts, achieving good results after 5000 trajectory samples. The evaluation compares these methods with Koopman, neural-network, and nonlinear-control baselines.
- Sample efficiency: DKUC and DKAC performance is not sensitive to training-sample count compared with KDNN and KRBF.The DampingPendulum evaluation uses 200, 1000, 5000, 20000, and 50000 trajectory samples.
- Sample efficiency: 5000 trajectory samples are sufficient for the proposed methods to achieve good prediction results.Each trajectory contains 15 time steps, and 5000 samples correspond to 25 minutes of data.
- Evaluation metrics: The prediction evaluation reports mean log10 maximum error over 30-step horizons and errors at the 15th step.The 30-step results are plotted in Fig. 4, while 15th-step results are reported in Table I.
1) Classical Control:
DKUC and DKAC control all evaluated classical systems, while DKAC is especially effective when the control term is highly nonlinear and state-dependent. The experiments also examine total cost across initial states and robot tracking under measurement noise.
- Classical control: DKUC and DKAC succeed in controlling all evaluated classical systems and achieve good performance.The comparison includes Koopman baselines and iLQR; out-of-range control costs are marked as NaN.
- Classical control: DKAC outperforms DKUC and KDerivative on environments such as DampingPendulum, where the control term is highly nonlinear and state-dependent.Only DKAC succeeds in controlling the damping pendulum to the zero point.
- Initial-state analysis: DKAC directs the DampingPendulum to the desired state with less cost for most initial conditions, whereas other methods fail when the initial state is far away.Total cost is evaluated over the corresponding initial states.
- Franka control: In the Franka experiment, the desired joint velocity is the control input at 100Hz, while PID computes actual force at 1000Hz.The tracking target is the desired end-effector trajectory.
- Franka control: The proposed algorithm achieves the lowest Franka tracking error and remains robust to Gaussian measurement noise.Only the proposed algorithm succeeds on the noised Franka-Star task.
VI. LIMITATION AND FUTURE DIRECTION
The framework achieves strong prediction and control performance, but its inverse control decoding, scalability, and handling of input constraints remain open issues.
- Limitations: The DKN algorithm achieves the best prediction performance, but recovering real control signals from its auxiliary control network lacks a unique inverse mapping.Designing an invertible decoder is identified as a key problem, with autoencoders and invertible neural networks suggested as possible approaches.
- Future directions: Extending the approaches to higher-dimensional nonlinear systems, particularly soft robots with highly complex dynamics, is identified as promising future work.
- Limitations: The approach does not consider input and state constraints, and the control network distorts input constraints in the embedding space.State constraints are easy to derive because the embedding retains the original state, whereas input constraints require further treatment.
- Supported outcome: Experimental results show improved prediction and control performance, including control of systems with highly nonlinear control terms.