PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-Step Prediction of Dynamic Systems with Recurrent Neural Networks

Nima Mohajerin, Steven L. Waslander

arXiv:1806.00526v1cs.NEcs.RO

TL;DR

Long-horizon prediction is challenged by errors from unmodeled dynamics and by the need to initialize RNN states appropriately. The paper applies NN-based initialization to recurrent models of two aerial vehicles and combines a simplified quadrotor motion model with RNNs. NN-based initialization improves prediction accuracy over washout, while the hybrid quadrotor model reduces errors relative to the black-box model at the shortest reported horizon.

  • Problem

    Multi-step prediction errors can grow over long horizons, and RNN prediction depends on proper state initialization.

    Source: §I · pp. 1–2

  • Method

    The study uses NN-based state initialization for RNN models of a helicopter and quadrotor, and combines a simplified quadrotor motion model with RNNs.

    Source: §I · pp. 2, 10, §V · p. 6, §VII · p. 11

  • Results

    NN-based initialization improves immediate and overall prediction accuracy over washout; at 0.4s, the hybrid model reduces body rate error by 50% and velocity error by about 20× versus the black-box model.

    Source: E. NN-based Initialization vs. Washout · p. 8, J. Hybrid Mmodel of the Quadrotor · p. 10

  • Takeaways & Limitations

    The hybrid model’s multi-step prediction accuracy is demonstrated experimentally, and its predictions can be used in a variety of control schemes.

    Source: §VII · p. 11

Abstract

from arXiv · show

Recurrent Neural Networks (RNNs) can encode rich dynamics which makes them suitable for modeling dynamic systems. To train an RNN for multi-step prediction of dynamic systems, it is crucial to efficiently address the state initialization problem, which seeks proper values for the RNN initial states at the beginning of each prediction interval. In this work, the state initialization problem is addressed using Neural Networks (NNs) to effectively train a variety of RNNs for modeling two aerial vehicles, a helicopter and a quadrotor, from experimental data. It is shown that the RNN initialized by the NN-based initialization method outperforms the state of the art. Further, a comprehensive study of RNNs trained for multi-step prediction of the two aerial vehicles is presented. The multi-step prediction of the quadrotor is enhanced using a hybrid model which combines a simplified physics-based motion model of the vehicle with RNNs. While the maximum translational and rotational velocities in the quadrotor dataset are about 4 m/s and 3.8 rad/s, respectively, the hybrid model produces predictions, over 1.9 second, which remain within 9 cm/s and 0.12 rad/s of the measured translational and rotational velocities, with 99\% confidence on the test dataset

I. INTRODUCTION

The paper studies RNN multi-step prediction, emphasizing that initialization and unmodeled dynamics affect accuracy over longer horizons. It introduces NN-based initialization and examines recurrent models of two aerial vehicles, including a hybrid quadrotor model.

I. INTRODUCTION

  • Unmodeled dynamics can cause prediction errors to grow over long horizons, while RNN predictions depend heavily on their initial state.Washout’s duration is difficult to determine, may destabilize training, and leaves its input sequence unused; that inefficiency matters when experimental data are costly.Source: §I · pp. 1–2
  • Multi-step prediction can support MPC with slower updates and help estimate vehicle position and speed during temporary measurement outages.The work also demonstrates using RNNs to predict two robotic systems many steps ahead from input information.Source: §I · pp. 1–2
  • First-principles models require identified physical parameters and may omit difficult effects, whereas black-box models learn nonlinearities from input-output observations.The paper cites the quadrotor vortex-ring effect as an example of a property that is difficult to model explicitly.Source: §I · p. 1
  • The NN-based initializer significantly outperforms washout initialization, and the study models a helicopter and quadrotor from experimental data while revisiting a hybrid quadrotor model.The authors frame the work as demonstrating neural networks’ ability to learn unmodeled robotic-system dynamics for multi-step prediction.Source: §I · pp. 1–2

II. BACKGROUND AND LITERATURE REVIEW

  • RNNs are suitable for multi-step prediction because they exhibit dynamics, and theory links their trajectory approximation to proper initialization.By contrast, feed-forward networks lack dynamics and are mainly used as single-step predictors or compensators.Source: §II · p. 2
  • NARX-MLPs can lose recurrent dynamics under teacher forcing, while closed-loop training can face gradient-stability problems.Prior work also reports NARX-MLPs outperforming RMLPs in single-step prediction of a real helicopter.Source: §II · p. 2

III. PROBLEM FORMULATION

The formulation treats a discrete-time nonlinear system as producing an output sequence in response to an input sequence. It defines multi-step prediction as choosing an RNN and initial state to minimize error over a prediction interval, with practical training based on measured sequences.

A. Multi-Step Prediction with Recurrent Neural Networks

  • For a general nonlinear system in discrete time, multi-step prediction estimates the output over the same horizon as a given input sequence.The estimate is assessed using a sum-of-squares error over prediction length T.Source: §III · p. 3, A. Multi-Step Prediction with… · p. 3
  • The system’s input and output are represented as measurable sequences, and the dataset consists of input-output tuples of equal length.The system response to an input sequence is denoted as an output sequence.Source: A. Multi-Step Prediction with… · pp. 3–4
  • The RNN maps each input and its current state to an output and an updated state, using network parameters that depend on the architecture.The output map is linear for this regression problem; the state-update function may be explicit for RMLPs or implicit for LSTMs.Source: A. Multi-Step Prediction with… · p. 3
  • RNN states include output feedback and internal feedback, and the state vector collects these buffered values.The discrete-time RNN evolves by updating its state and generating an output at each time step.Source: A. Multi-Step Prediction with… · p. 3
  • The objective is an RNN that minimizes mean SSE over the prediction interval for input sequences.The formulation expresses the RNN output sequence as a function of its initial state and the input sequence.Source: A. Multi-Step Prediction with… · p. 3
  • Because optimization over all possible input-output sequences is impractical, training uses numerical optimization on a collected dataset.The dataset contains D time-series samples.Source: A. Multi-Step Prediction with… · p. 4
  • The paper uses some RNN training techniques and refers readers to prior discussions for details.The techniques are described in Section VI, with references to and.Source: A. Multi-Step Prediction with… · p. 4

B. State Initialization in Recurrent Neural Networks

An RNN’s initial state shapes its immediate response, so state initialization is part of obtaining accurate multi-step predictions. The RNN can be viewed as mapping an initial state and input sequence to an output sequence.

B. State Initialization in Recurrent Neural Networks

IV. SOLUTIONS TO RNN STATE INITIALIZATION PROBLEM

The formulation uses a linear output map and treats state initialization as an optimization problem during system identification. It reviews washout, joint optimization of initial states and network weights, and NN-based initialization.

IV. SOLUTIONS TO RNN STATE INITIALIZATION PROBLEM

  • The RNN output is generated by a linear mapping of the network state and input.The output-layer weights are included in the network weight vector.Source: §IV · p. 4
  • During training, state initialization for system identification is framed as minimizing a cost over initial states because the true system model is not necessarily known.The formulation uses an RNN approximation to the modeled system and its prediction error at the interval’s start.Source: §IV · p. 4
  • The initial hidden state is constrained to its valid range, and an exact initialization solution may overfit when initialization must also work at test time.The paper studies washout, joint optimization of initial states and weights, and NN-based initialization.Source: §IV · p. 4

A. RNN State Initialization: Washout

The washout method runs an RNN from zero or random states until their initial effect fades. The optimization alternative augments the trainable weights with initial states but lacks a general post-training mechanism for generating them.

A. RNN State Initialization: Washout

  • Washout initializes states to zero or random values, then runs the RNN until the initial-state effect wears off.The washout period has no deterministic selection procedure and is treated as a hyperparameter, reducing learning speed.Source: A. RNN State Initialization: Washout · p. 4
  • Running an unstable RNN during training can produce very large costs and a divergent learning curve.The washout period may also involve temporary instability, despite the RNN's overall stability being linked to the system being learned.Source: A. RNN State Initialization: Washout · p. 4

B. RNN State Initialization: Optimization

  • Augmenting the trainable weights with initial states does not provide a general way to generate those states after training.Problem-specific optimization may recover initial hidden states in special cases, but the approach is not architecture-general.Source: B. RNN State Initialization:… · p. 5

C. RNN State Initialization: NN-based

The NN-based approach trains an initializer to provide the predictor's initial states from a measurement history. It supports both static MLP and recurrent initializer designs, trained together with the predictor.

C. RNN State Initialization: NN-based

  • An auxiliary neural network maps a short history of system inputs and outputs to the predictor's initial states.The initializer approximates the mapping needed to produce the predictor state values.Source: C. RNN State Initialization: NN-based · p. 5
  • The training sample is split into an initialization segment of length τ and a prediction segment of length T, which together train the initializer-predictor pair.The initialization segment feeds the initializer, while the prediction segment trains the pair.Source: C. RNN State Initialization: NN-based · p. 5, Figure 1 · p. 5
  • An MLP initializer approximates predictor-state dynamics over a fixed initialization interval as a static function of the measurement history.The MLP receives a history of system measurements and produces the predictor's initial states.Source: C. RNN State Initialization: NN-based · p. 5
  • A recurrent initializer models predictor-state dynamics with a sequence-to-sequence network that processes measurement history over τ.For the RNN-RNN pair, the initializer's states are set to zero, and the initialization segment must capture predictor-state dynamics.Source: C. RNN State Initialization: NN-based · p. 5
  • The initializer and predictor are trained together by taking gradients of their total cost with respect to their augmented weights.The cost combines prediction error and a penalty on the initializer output, balanced by coefficients α and β, set to one in this work.Source: C. RNN State Initialization: NN-based · p. 5

V. MODEL ARCHITECTURES

The paper considers black-box RNN predictors with either MLP or LSTM initializers, as well as a hybrid class that combines an RNN with a physics-based model. The black-box predictors include MLFC RNNs and layered LSTM cells.

V. MODEL ARCHITECTURES

  • The black-box class pairs an MLFC or layered LSTM predictor with an MLP or LSTM initializer.The alternative model class is an end-to-end hybrid of a black-box model and a simple physics-based dynamic model.Source: §V · p. 6

A. Multi-Layer-Fully-Connected RNN

  • MLFC RNNs use sigmoid layers connected in series with feedforward and feedback skip connections.The skip connections help attenuate the structural vanishing-gradient problem.Source: A. Multi-Layer-Fully-Connected RNN · p. 6

B. Long-Short-Term-Memory RNN

  • The LSTM used here has peephole connections, and both its cell states and hidden states are initialized by the initializer network.For regression, the output activation is identity and the cell activation is tangent-hyperbolic.Source: B. Long-Short-Term-Memory RNN · p. 6

C. Hybrid model

The hybrid architecture combines a physics model with RNN input and output modules to model known dynamics and compensate for model error. For the quadrotor, the physics module implements a motion model; the dataset covers varied indoor flight regimes.

C. Hybrid model

  • The hybrid uses an input model to estimate white-box inputs, a physics model to update states, and an output model to compensate for prediction error.The input model and output model are RNNs with initialization networks; the physics-model output is included in the hybrid prediction.Source: C. Hybrid model · pp. 6–7
  • For the quadrotor, the physics module is a motion model that receives torques and thrust and updates vehicle states.The state vector includes Euler angles, body angular rates, position, and inertial-frame velocity.Source: C. Hybrid model · p. 7

A. Quadrotor Dataset

  • The quadrotor dataset contains about 3 hours 50 minutes of indoor flight, or about 1.4 million time steps per time-series, spanning hover through aggressive maneuvers.States were measured with onboard sensors and a motion-capture system; maximum values for six degrees of freedom are listed in Table I.Source: A. Quadrotor Dataset · p. 7, Table I · p. 7

B. Helicopter Dataset

The helicopter experiments compare dataset and RNN setup choices, evaluation metrics, and NN-based initialization against washout. NN-based initialization improves immediate and overall prediction accuracy, though the methods eventually converge to similar error.

B. Helicopter Dataset

  • The helicopter dataset contains about 55 minutes of outdoor flight and 300k samples per quantity, without wind measurements.It was collected for Apprenticeship Learning and has also been used for single-step prediction and system identification.Source: B. Helicopter Dataset · p. 7

C. Architectures and Implementation

  • Predictors are MLFCs, LSTMs, or LSTMs with TDL buffers, paired with washout, MLP, or LSTM initialization.All networks use a 0.1-second initialization interval and are evaluated at prediction horizons of 0.4 and 1.9 seconds.Source: C. Architectures and Implementation · pp. 7–8

D. Evaluation Metrics

E. NN-based Initialization vs. Washout

  • NN-based initialization improves immediate and overall prediction accuracy over washout, although both methods eventually converge to almost the same error.The comparison is on simplified helicopter data; the authors also note that choosing an efficient washout period is difficult.Source: E. NN-based Initialization vs. Washout · p. 8, Figure 5 · p. 8

F. MLFC vs. LSTM

The helicopter experiments compare MLFC and LSTM predictors, then examine LSTM variants with different initializers and configurations. MLP-initialized LSTMs outperform other methods, including MLFCs with more weights, while the tested models behave similarly across the later comparisons.

F. MLFC vs. LSTM

  • LSTMs with MLP initialization outperform other methods on RMSSE, and LSTMs with fewer weights perform better than MLFCs.The RMSSE-versus-network-size comparison uses the reduced helicopter dataset.Source: F. MLFC vs. LSTM · p. 8, Figure 6 · p. 9

G. MLP vs RNN Initializers

  • The LSTM variants map pilot commands to Euler rates, with optional tapped-delay buffers at the input and output.The network layers are connected in series, with the final layer output fed back to the first.Source: G. MLP vs RNN Initializers · p. 9
  • The tested models behave similarly on test-set total RMSSE across the 0.4- and 1.9-second prediction horizons.The comparison spans network sizes and types; the authors focus on four architectures based on these results.Source: G. MLP vs RNN Initializers · p. 9, Figure 7 · p. 9

H. Black-box Modeling of the Helicopter

The helicopter results show non-monotonic long-horizon error behavior and data limitations, while the quadrotor black-box models separate body-rate and velocity prediction. Quadrotor body-rate prediction remains accurate over long horizons, but practical-mode velocity prediction is substantially worse than teacher-forced prediction.

H. Black-box Modeling of the Helicopter

  • Helicopter prediction error generally rises through the 0.4-second horizon, but at longer horizons it peaks early and then attenuates.Figure 8 reports mean error distributions, with velocity in the top row and angular rates in the bottom row.Source: H. Black-box Modeling of the Helicopter · p. 9, Figure 8 · p. 9
  • Helicopter results are constrained by a relatively small dataset, outdoor flights with unreported wind, and pilot-command inputs that introduce transformations and timing delays.The authors suggest actual motor speeds may mitigate the command-delay and transformation effects, and prefer a controlled environment.Source: H. Black-box Modeling of the Helicopter · pp. 9–10

I. Black-box Modeling of the Quadrotor

  • The quadrotor model decouples body-rate prediction from velocity prediction, then feeds predicted body rates into the velocity network in practical mode.During training, the velocity network receives measured body rates, resembling teacher forcing; deployment instead cascades the two networks.Source: §I · p. 10
  • Over 1.9 seconds, quadrotor body-rate accuracy remains better than 3.5 deg/sec on average, while teacher-forced velocity error remains below 4 cm/s over almost 2 seconds.The RNN-initialized LSTMs perform better over longer prediction lengths; TDLs improve prediction accuracy.Source: §I · p. 10, Figure 9 · p. 10
  • Practical-mode quadrotor velocity error is approximately 25× worse than teacher-forced prediction, leaving velocity prediction far from desirable.The black-box model provides reliable and accurate body-rate prediction, but the practical-mode velocity prediction is degraded.Source: §I · p. 10

J. Hybrid Mmodel of the Quadrotor

The quadrotor hybrid model combines a simplified motion model with initialized RNN modules and improves short-horizon errors over the black-box model. At 1.9 seconds, the comparison depends on the output: black-box body-rate prediction becomes better later, while its velocity prediction remains worse.

J. Hybrid Mmodel of the Quadrotor

  • The motion model contributes to early prediction accuracy, but may restrict the search for optimal output-module weights.The authors suggest this restriction may explain why black-box body-rate prediction becomes better later.Source: J. Hybrid Mmodel of the Quadrotor · pp. 10–11
  • At the 1.9-second horizon, black-box velocity prediction remains worse than hybrid prediction, though its error decreases slightly later.Figure 12 compares velocity error distributions for the two models.Source: J. Hybrid Mmodel of the Quadrotor · p. 11, Figure 12 · p. 11
  • The motion-model output begins with non-zero error, so the output module must immediately compensate and requires proper RNN state initialization.Compensated states are fed back into the motion model at each prediction step.Source: J. Hybrid Mmodel of the Quadrotor · p. 11
  • At a prediction horizon of 0.4 seconds, the hybrid model’s velocity prediction is over an order of magnitude better than the black-box model’s, as shown by the error distributions.Figure 11 compares body-rate error distributions for the hybrid and black-box models at the same prediction horizon.Source: Figure 10 · p. 11, Figure 11 · p. 11
  • At a prediction length of 1.9 s, Figure 13 compares the body-rate error distributions of the hybrid and black-box models.Source: Figure 13 · p. 11

VII. CONCLUSION

  • The hybrid model’s multi-step prediction accuracy is demonstrated experimentally, and the authors state it can be employed in a variety of control schemes.Source: §VII · p. 11
Loading 1806.00526v1…