Source-linked AI summary
Backprop KF: Learning Discriminative Deterministic State Estimators
Tuomas Haarnoja, Anurag Ajay, Sergey Levine, Pieter Abbeel
TL;DR
Generative state estimators struggle with rich sensory observations, while discriminative latent-state models are harder to train. The paper reformulates state estimation as a deterministic computation graph trained end-to-end, and reports improvements over standard generative estimators and general-purpose recurrent networks.
Problem
Generative models struggle with high-dimensional observations such as camera images, while discriminative latent-variable estimators are more complex to train with partial state labels.
Method
The method reformulates probabilistic state estimation as a deterministic computation graph, using backpropagation through time to train a Kalman filter with a convolutional network end-to-end.
Results
The approach significantly improves state estimation over standard generative methods and recurrent neural networks, with BKF outperforming KF-based estimators and LSTMs in the reported experiments.
Takeaways & Limitations
Filtering structure provides an inductive bias that helps discriminative state estimators, particularly when training data are limited.
Takeaways & Limitations
The experiments primarily use Kalman-filter models, although the approach requires a differentiable filter mapping and future work targets more complex dynamics and larger latent states.
Abstract
from arXiv · showhide
Generative state estimators based on probabilistic filters and smoothers are one of the most popular classes of state estimators for robots and autonomous vehicles. However, generative models have limited capacity to handle rich sensory observations, such as camera images, since they must model the entire distribution over sensor readings. Discriminative models do not suffer from this limitation, but are typically more complex to train as latent variable models for state estimation. We present an alternative approach where the parameters of the latent state distribution are directly optimized as a deterministic computation graph, resulting in a simple and effective gradient descent algorithm for training discriminative state estimators. We show that this procedure can be used to train state estimators that use complex input, such as raw camera images, which must be processed using expressive nonlinear function approximators such as convolutional neural networks. Our model can be viewed as a type of recurrent neural network, and the connection to probabilistic filtering allows us to design a network architecture that is particularly well suited for state estimation. We evaluate our approach on synthetic tracking task with raw image inputs and on the visual odometry task in the KITTI dataset. The results show significant improvement over both standard generative approaches and regular recurrent neural networks.
1 Introduction
State estimation matters for mobile robotics, but generative models struggle with high-dimensional observations and discriminative latent-state training is difficult. The paper addresses this by training a deterministic, recurrent-style computation graph end-to-end.
- Generative state estimators struggle with camera images because they must model the full sensor-reading distribution.
- Discriminative state estimators avoid modeling sensor distributions but are more complex to train with latent variables and partial labels.
- The proposed method replaces probabilistic latent-variable inference with an equivalent deterministic computation graph optimized by backpropagation and gradient descent.
- The architecture supports nonlinear components, including convolutional processing of raw camera images, trained jointly with observation and transition functions.
- Evaluations on visual tracking and KITTI vehicle-pose estimation show significant improvement over standard generative methods and recurrent neural networks.
2 Related Work
Related work spans probabilistic filters, discriminative state models, and recurrent neural networks. The paper connects these traditions by using filtering structure to design trainable recurrent computation graphs for state estimation.
- Probabilistic state-space models use filtering and smoothing, including Kalman, unscented, and particle filters, for state estimation.
- Generative models become impractical for highly dimensional, nonlinear observations such as camera images, despite complex observation models and approximate inference.
- Discriminative CRFs and conditional state-space models directly estimate state conditionals, while later work adds neural-network nonlinearities.
- Prior RNN state-estimation methods generally addressed simple tasks and often omitted complex sensory inputs such as images.
- The proposed approach treats the estimator as a deterministic computation graph and uses filtering knowledge to improve recurrent architectures over black-box alternatives.
3 Preliminaries
The standard pipeline first extracts a low-dimensional signal from high-dimensional observations, then filters that signal to infer the desired state. Its intermediate predictor is not optimized directly for final estimation accuracy.
- Direct generative filtering of camera images is difficult because images arise from complex, highly nonlinear processes.
- A feedforward model gθ(ot) can predict a low-dimensional signal zt, which a Kalman filter uses to infer hidden states and desired labels.
- The piecewise predictor avoids explicit observation-distribution modeling by conditioning on raw observations while treating xt as an internal latent variable.
- Because gθ(ot) is trained to predict an intermediate zt rather than final state-estimation accuracy, the standard pipeline is not end-to-end optimized for the task.
4 Discriminative Deterministic State Estimation
The paper treats a state estimator as a deterministic computation graph, enabling end-to-end discriminative training by backpropagating sequence loss through the filter. This connects filtering architectures with recurrent neural networks while retaining nonlinear observation functions.
- The observation function gθ(ot) is trained discriminatively over entire sequences rather than independently at individual time steps.The filter is represented as a computation graph whose output distribution receives a sequence-level loss.
- The filter computes its next internal state as st+1 = κ(st, zt+1), and loss gradients are recursively propagated backward through these states.The recursion computes gradients with respect to filter states from back to front before applying the chain rule.
- All required derivatives are obtained from the observation function, filter update κ, output function q, and loss function.These components define the local operations needed for differentiating the computation graph.
- The resulting gradients optimize θ with gradient descent, implementing backpropagation through time for this recurrent state-estimation computation graph.The paper identifies this procedure as BPTT, a standard algorithm for training recurrent neural networks.
- The architecture spans discriminatively trained filters and general recurrent neural networks, using a Kalman update with a nonlinear convolutional observation function in experiments.This design combines a standard Kalman filter update with a CNN that processes observations.
5 Experimental Evaluation
The evaluation compares BKF with feedforward, piecewise-KF, and LSTM estimators on synthetic visual tracking and KITTI visual odometry. BKF achieves the lowest error across the reported comparisons while using Kalman-filter transitions and fewer parameters than LSTM models.
- 5.1 State Estimation Models: BKF combines a convolutional network that maps raw images to observations and covariances with a recurrent Kalman-filter component.The network produces z_t and R_t, while the filter integrates observations over time.
- 5.3 Synthetic Visual State Estimation Task: The synthetic task tracks a red disk under clutter, noise, and occlusion, with difficulty controlled by the number of distractor disks.The evaluation includes sequences where the target is frequently occluded.
- 5.3 Synthetic Visual State Estimation Task: BKF outperforms feedforward, piecewise-KF, and LSTM estimators on the synthetic benchmark.The piecewise KF relies heavily on dynamics because its observation covariance is not conditioned on the input, whereas BKF learns input-conditioned covariances.
- 5.3 Synthetic Visual State Estimation Task: BKF achieves the lowest error in nearly all synthetic test conditions and uses dramatically fewer parameters than LSTM models.The parameter reduction comes from using simple Kalman-filter updates for transitions.
- 5.4 KITTI Visual Odometry Experiment: For KITTI visual odometry, the models estimate relative vehicle-pose changes from image sequences, using convolutional processing of image differences.The dataset contains 11 trajectories and is challenging because it is small and visually diverse.
- 5.4 KITTI Visual Odometry Experiment: BKF outperforms the other variants for both vehicle position and heading, while Kalman-filter variants generalize slightly better than LSTM to mixed sequence lengths.LSTM performance degrades faster with fewer training sequences and generally fails to generalize to longer sequences.
6 Discussion
The paper reformulates probabilistic state estimation as deterministic recurrent computation graphs trained with backpropagation, enabling expressive discriminative models for complex observations. Experiments show gains over piecewise approaches and general-purpose LSTMs when data are limited, while the approach extends beyond Kalman filters.
- Method: The method reformulates probabilistic generative state estimation as a deterministic computation graph trained end-to-end with backpropagation through time.The resulting model retains the representational power of the corresponding probabilistic estimator while using stochastic gradient descent for parameter learning.
- Method: It incorporates nonlinear observation and transition functions, including convolutional networks processing raw camera images.The entire network is optimized jointly for the state estimation task.
- Results: End-to-end discriminative training substantially improves performance over a standard piecewise pipeline that integrates learned observations with a generative filter.The comparison concerns training the discriminative model jointly with the filter versus training it separately to produce intermediate observations.
- Results: BKF can match the accuracy of a large LSTM and outperforms a general-purpose LSTM when the dataset is limited.The paper attributes this behavior to filter-informed architectural structure that supplies a better inductive bias in data-limited settings.
- Scope: The computation-graph construction applies to differentiable probabilistic filters beyond the Kalman filter, including information, unscented Kalman, and particle filters.The stated condition is that update equations can be expressed as a differentiable mapping from observations and the previous state to the new state.
A Backprop KF Computation Graph Architecture
The BKF computation graph combines a feedforward observation network with a recurrent Kalman-filter component and differentiable parameterizations for valid covariance matrices. Its inputs, updates, initialization, and training labels are assigned distinct roles in the graph.
- Architecture: The BKF architecture combines feedforward neural-network blocks, recurrent Kalman-filter blocks, and a block enforcing positive semidefiniteness of the observation covariance.The covariance is parameterized through a Cholesky decomposition with exponentiated diagonal elements.
- State and observation model: The Kalman filter is defined over a dynamical system whose observation is inferred from the input through z_t = g_θ(o_t), while y_t denotes the ground-truth observation.The supplied passage identifies the learned observation mapping and distinguishes inferred observations from ground truth.
- State and observation model: The state vector tracks system position and velocity, with linear dynamics represented by f(x_t) = Ax_t and Gaussian process and observation noise covariances Q and R_t.The noise terms are assumed IID and zero mean; R_t is parameterized as L_tL_t^T.
- Recurrence: Kalman-filter updates form the recurrent computation that performs covariance and observation updates from one state to the next.The section presents these updates as the recurrence underlying the computation graph.
- Recurrence: The extended Kalman filter is obtained by setting A to the Jacobian of the dynamics function at the current state estimate.This linearizes nonlinear dynamics within the filter recurrence.
- Initialization and training: The initial state x_0 is treated as a task-dependent hyperparameter, while labels y_t enter through the training cost rather than the Kalman-filter updates.The labels are noiseless observations used at training time.
B Synthetic Tracking Experiment
The synthetic tracking experiment estimates a moving red disk’s position and velocity from third-person images using a feedforward network coupled to a Kalman-filter computation graph. Initialization and training procedures are specified alongside the feedforward architectures.
- Task: The tracking state contains the red disk’s 2D position and velocity, using a simple integrator dynamics model with noise applied only to velocity.This defines the synthetic task’s latent state and dynamics assumptions.
- Model: The feedforward network receives a third-person-view image as observation o_t and supplies inputs to the filter state variables.The experiment uses image observations to drive the BKF computation graph.
- Initialization: The filter state variables are initialized with the ground-truth state and identity matrix to suppress a large initial-estimation transient.The initialization is selected specifically for the tracking experiment.
- Training: Training uses ADAM with manually selected learning rates, and the piecewise KF is initialized from a pretrained feedforward model before further fine-tuning.The passage describes sequential pretraining and subsequent model refinement.
- Architecture: The feedforward networks use convolutional layers, pooling, normalization, fully connected layers, and outputs for inferred observations and covariance parameters.The listed architectures include image inputs and outputs for z_t and the parameterized lower-triangular covariance factor.
C KITTI Visual Odometry
The KITTI visual odometry experiment uses a five-variable vehicle state, velocity observations, and position-heading ground truth within a nonlinear dynamics model. Training and filter variants use task-specific pretraining and learned covariance parameters.
- Task formulation: The visual-odometry state has five variables: inertial-frame position and heading plus ego-centric forward and angular velocities.The experiment distinguishes the coordinate frames used for pose and velocity variables.
- Task formulation: The observation z_t contains vehicle velocities, whereas the ground-truth observation y_t contains position and heading.The state-estimation task combines velocity inputs with pose targets.
- Dynamics: The nonlinear dynamics model is linearized at the current state estimate for the Kalman-filter variants.This produces the local linearization used by the visual-odometry filter.
- Training: The feedforward network is pretrained to predict velocities, and the additional maximum-likelihood pretraining step is omitted for the visual-odometry experiment.The training procedure differs from the synthetic tracking setup in this respect.
- Training: Kalman-filter variants additionally learn the dynamics covariance, while the LSTM recurrent layers are also pretrained.These are experiment-specific training choices for the compared model families.