PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Neural Networks with Koopman Operators for Modeling and Control of Autonomous Vehicles

Yongqian Xiao, Xinglong Zhang, Xin Xu, Xueqing Liu, Jiahang Liu

arXiv:2007.02219v2eess.SY

TL;DR

Precise vehicle dynamics models are difficult to obtain because autonomous vehicles exhibit strong nonlinearity and uncertainty, while existing machine-learning models have limited generalization and interpretability. The paper proposes Deep EDMD to learn a finite-dimensional Koopman model in a lifted linear space and designs DE-MPC for path tracking. CarSim simulations report high modeling precision across a wide operating range, improved modeling performance over prior methods, and effective path tracking.

  • Problem

    Precise vehicle dynamics modeling is difficult under strong nonlinearity and uncertainty, and existing machine-learning models still have limited generalization and interpretability.

  • Method

    Deep EDMD uses deep neural networks within EDMD to learn a finite-dimensional Koopman representation, which is then used to design DE-MPC for autonomous-vehicle path tracking.

  • Results

    CarSim simulations show high modeling precision across a wide operating range, improved modeling performance over EDMD, DNN, and ELM-EDMD, and effective path tracking.

  • Takeaways & Limitations

    The learned Koopman model supports a data-driven MPC that represents nonlinear vehicle dynamics through linear prediction in a lifted feature space.

Abstract

from arXiv · show

Autonomous driving technologies have received notable attention in the past decades. In autonomous driving systems, identifying a precise dynamical model for motion control is nontrivial due to the strong nonlinearity and uncertainty in vehicle dynamics. Recent efforts have resorted to machine learning techniques for building vehicle dynamical models, but the generalization ability and interpretability of existing methods still need to be improved. In this paper, we propose a data-driven vehicle modeling approach based on deep neural networks with an interpretable Koopman operator. The main advantage of using the Koopman operator is to represent the nonlinear dynamics in a linear lifted feature space. In the proposed approach, a deep learning-based extended dynamic mode decomposition algorithm is presented to learn a finite-dimensional approximation of the Koopman operator. Furthermore, a data-driven model predictive controller with the learned Koopman model is designed for path tracking control of autonomous vehicles. Simulation results in a high-fidelity CarSim environment show that our approach exhibit a high modeling precision at a wide operating range and outperforms previously developed methods in terms of modeling performance. Path tracking tests of the autonomous vehicle are also performed in the CarSim environment and the results show the effectiveness of the proposed approach.

I. INTRODUCTION

The paper addresses the difficulty of obtaining precise, interpretable vehicle dynamics models for control under nonlinear and uncertain conditions. It proposes Deep EDMD with an interpretable Koopman operator and uses the learned model in a path-tracking MPC.

  • Vehicle dynamics are difficult to model precisely because longitudinal and lateral motion are coupled, while nonlinearity and uncertainty intensify near tire-road friction limits.
  • Deep neural-network vehicle models can lack interpretability, exhibit sensitivity to uncertainties, and be difficult to use in well-posed controllers such as MPC.
  • CarSim simulations compare the approach with EDMD, DNN, and ELM-EDMD for modeling and evaluate its path-tracking control performance.
  • Deep EDMD learns observable basis functions and feature representations automatically within EDMD, producing a linear global model in a lifted space with nonlinear mappings to the original state.
  • The proposed DE-MPC uses the learned model for time-varying reference tracking, using its linear component for prediction and the nonlinear mapping to reset the lifted state.

III. THE BASIC MODEL DESCRIPTION OF AUTONOMOUS VEHICLES

The paper models a front-wheel-steering vehicle with four wheels and no roll motion using tire forces, aerodynamic effects, and Newton–Euler equations. Because strong nonlinearity is embedded in the resulting dynamics, the authors motivate a lifted linear representation.

  • The vehicle model assumes four wheels, front-wheel steering, and planar motion without roll, with tire forces and moments represented in the vehicle sketch.
  • Longitudinal and lateral tire forces are defined for each wheel, with front-wheel steering and zero rear-wheel angles represented by δ3 = δ4 = 0.
  • Aerodynamic forces depend on air density, frontal area, relative air speed, and yaw-angle-dependent drag coefficients.
  • The Pacejka tire model relates forces to tire load, longitudinal slip, sideslip angle, steering angle, and throttle or braking input.
  • The resulting equations describe longitudinal and lateral vehicle motion using velocities, mass, yaw inertia, yaw angle, slip angle, and steering variables.
  • Strong nonlinearity hidden in the vehicle dynamics motivates representing the global dynamics with a linear model in a lifted abstract state space.

IV. THE DEEP EDMD APPROACH FOR VEHICLE DYNAMICS MODELING

The Deep EDMD approach combines Koopman-operator modeling with deep neural networks to automatically construct a finite-dimensional observable subspace for vehicle dynamics.

  • Deep EDMD uses a deep neural network to automatically construct an observable subspace of the Koopman operator for vehicle-dynamics modeling.

A. The Koopman operator using EDMD

The Koopman operator represents nonlinear vehicle dynamics through linear evolution in a lifted observable space. EDMD approximates this operator using finite basis functions and least-squares estimation.

  • The Koopman operator maps observables of the nonlinear system forward through composition with the system dynamics.
  • Control inputs are incorporated by augmenting the state with the input and its shifted sequence.
  • EDMD represents observables with basis functions and computes a finite-horizon Koopman approximation using least squares.
  • For forced vehicle dynamics, the lifted model uses matrices A and B for the lifted state and control input, with C mapping lifted states toward outputs.
  • The matrices for the finite-dimensional model are obtained from snapshot data through least-squares residual minimization.

B. The Deep EDMD Algorithm

Deep EDMD integrates neural-network encoder and decoder components with EDMD to learn an interpretable finite-dimensional Koopman model. Its training combines multi-step prediction, lifted-space, reconstruction, robustness, and regularization objectives.

  • Deep EDMD uses a deep neural network to automatically construct an observable subspace and estimate a finite-dimensional Koopman operator.
  • The learned lifted dynamics propagate encoded states linearly through matrices A and B, with control inputs included in the lifted representation.
  • The encoder maps vehicle states into a lifted observable space, while the decoder reconstructs the original state from that space.
  • Deep EDMD minimizes multi-step prediction errors over a time window rather than relying only on one-step prediction.
  • Training combines lifted-space prediction, state reconstruction, infinite-norm robustness, and regularization losses in a batch-mode optimization procedure.
  • The algorithm samples state-control batches, computes multi-step lifted predictions and reconstructions, and updates network and linear-model parameters with Adam.

V. MPC WITH DEEP EDMD FOR TRAJECTORY TRACKING OF AUTONOMOUS VEHICLES

DE-MPC uses the learned Deep EDMD model as a linear predictor in a moving-horizon controller for time-varying trajectory tracking. The formulation penalizes tracking errors and control increments subject to input constraints.

  • DE-MPC uses the learned Deep EDMD dynamics to control a nonlinear vehicle system toward a time-varying reference.
  • The learned model is augmented by treating the input as an extended state, yielding a linear lifted model for prediction.
  • The optimization enforces constraints on both control inputs and their increments, with slack variables included in the formulation.
  • The MPC objective penalizes tracking errors and control increments using positive-definite matrices Q and R, with ρ penalizing slack variables.
  • The optimization is repeated at the next time step according to a moving-horizon strategy, while only the computed control action is applied.
  • At each time instant, the controller initializes the lifted state with the encoder and solves a finite-horizon optimization problem.

VI. SIMULATION AND PERFORMANCE EVALUATION

The proposed Deep EDMD method was evaluated in a high-fidelity CarSim simulation environment against EDMD, ELM-EDMD, and DNN baselines.

  • The evaluation compares Deep EDMD with EDMD, ELM-EDMD, and DNN in a high-fidelity CarSim simulation environment.

A. Data Collection

The study uses CarSim data from a C-Class sedan to build the vehicle-dynamics dataset for simulation-based evaluation.

  • Data were collected from 40 CarSim episodes, each containing 10,000–40,000 time steps at a 10 ms sampling period.The setup combined CarSim 2019 with Simulink/MATLAB 2017b and Logitech G29 steering and pedals.
  • The simulations used a C-Class sedan model as the original vehicle for data collection and evaluation.
  • Training hyperparameters for the simulations are summarized in Table I.

B. Data preprocessing and training

The training pipeline normalizes collected data, constructs sequence samples, and compares Deep EDMD with EDMD, ELM-EDMD, and DNN baselines. Deep EDMD uses learned neural feature representations within an EDMD framework.

  • Data preprocessing: About 7.7·10^5 snapshots from 40 episodes were split into 36 training, 2 validation, and 2 testing episodes.Adjacent state samples were concatenated, and sequence starting points were randomized before each training epoch.
  • Training setup: Deep EDMD used a five-layer encoder, a decoder, and lifted dimension L = 10 for learning vehicle dynamics.The encoder and decoder architectures were specified as [n 32 64 L L] and [L + n 128 64 32 n].
  • Baseline methods: EDMD constructed observables with thin plate spline radial basis functions, while ELM-EDMD replaced these kernel functions with extreme learning machines.
  • Baseline methods: The compared DNN used a fully trainable multilayer structure, while a second scenario added a randomly updated hidden layer and bias.
  • Training objective: Deep EDMD training minimized multi-step prediction loss, largest prediction error, and l2 weight regularization using weighted loss terms.The model was optimized with Adam; β1, β2, and β3 controlled the three loss components.

C. Performance evaluation

Deep EDMD produced accurate and robust vehicle-state predictions across operating conditions and supported effective DE-MPC path tracking. Its linear lifted model also enabled real-time-compatible computation in the tested CarSim scenarios.

  • Model prediction: Deep EDMD captured longitudinal and lateral velocity changes over a wide operating range, while EDMD and ELM-EDMD trajectories diverged after a short time.
  • Model prediction: DNN matched Deep EDMD for lateral velocity and yaw-rate prediction but produced larger longitudinal-velocity errors.
  • Training and validation: After 100000 batches, Deep EDMD training and validation losses reached 1.7·10^-4 and 1.9·10^-4, compared with DNN values of 2.0·10^-4 and 1.5·10^-3.
  • Robustness: With random hidden layers and biases, Deep EDMD had much smaller average prediction errors and error variations than DNN across 100 repeated predictions.
  • Robustness: Under random-layer training, Deep EDMD reconstruction errors converged to 7.6·10^-4 training and 7.3·10^-4 validation, whereas DNN errors fluctuated around 0.01 and 0.027.
  • Trajectory tracking: Tracking was more accurate and smoother with Np = 60, Nc = 50, while average computation times were 0.0047 s and 0.0092 s for the two cases.The reported times satisfy the real-time requirement for a 10 ms sampling interval.

VII. CONCLUSIONS

The paper presents a Deep EDMD vehicle model with an interpretable Koopman operator and a DE-MPC controller, reporting improved modeling performance and satisfactory CarSim tracking.

  • The approach uses encoder and decoder networks to learn Koopman eigenfunctions and modes, producing linear dynamic evolution with nonlinear static mapping for MPC.This structure is intended to support optimization-based real-time implementation.
  • Deep EDMD outperforms EDMD, ELM-EDMD, and DNN in modeling performance on high-fidelity CarSim data.The comparisons include modeling accuracy across the evaluated simulations.
  • DE-MPC achieves tracking results under prediction and control horizon settings of Np = 10, Nc = 7 and Np = 60, Nc = 50.The two horizon configurations were simulated for the tracking evaluation.
  • Satisfactory trajectory-tracking performance in CarSim further verifies the effectiveness of the Deep EDMD and DE-MPC methods.The paper identifies real-world autonomous-vehicle experiments as future research.
Loading 2007.02219v2…