PaperlayerView exact revision on arXiv

Source-linked AI summary

IMM-based Multiple Object Tracking using a State Prediction Neural Network

Chan-Bin Lim, Dong-Hee Paek, Seung-Hyun Kong

arXiv:2609.13307v1cs.ROcs.AI

TL;DR

Predefined motion models have limited ability to represent nonlinear road motion, while learned predictors and physics-based models offer complementary strengths. PR-IMM integrates a Transformer predictor and CV, CA, and CT modes, combining their priors according to mode probabilities. It performs best among the compared single models, conventional IMM, and standalone PR, improving position estimation and identity-preserving tracking.

  • Problem

    IMM models rely on predefined motion patterns and have limited ability to represent nonlinear object motion in real road environments.

    Source: §I · p. 1

  • Method

    PR-IMM integrates a Transformer predictor using radar Doppler measurements as an IMM mode alongside CV, CA, and CT, combining their prior states through mode probabilities.

    Source: §I · p. 1, 1) Input Mixing · p. 4

  • Results

    PR-IMM performs best among the CV, CA, and CT single models, conventional IMM, and standalone PR; versus standalone PR, IDSW drops from 915 to 519 and IDF1 rises from 0.8795 to 0.9020.

    Source: B. Quantitative Performance Comparison · p. 6, §V · p. 7

  • Takeaways & Limitations

    The results support integrating a learned state predictor into probabilistic IMM for improved position estimation and identity-preserving tracking under diverse motion and observation conditions.

    Source: §V · p. 7

Abstract

from arXiv · show

Object tracking is essential for autonomous vehicles to avoid obstacles and plan routes. Radar maintains detection performance even in adverse weather and can measure relative velocity through the Doppler effect, making it well suited for object tracking. In this paper, we propose a data-driven state PRedictor-based Interacting Multiple Model tracking method (PR-IMM) that improves nonlinear object-motion representation while preserving the stability and interpretability of physics-based motion models. The proposed method employs a transformer-based PRediction model (PR) that incorporates radar Doppler measurements to predict object displacement. The PR model is integrated into the IMM as a mode alongside the CV, CA, and CT motion models, and their prior positions are dynamically combined according to the mode probabilities. Experimental results show that PR-IMM reduces position-estimation error by 57.3% over the IMM and by 16.5% over the PR, while reducing ID switches by 25.3% and improving IDF1 by 9.6% over the IMM.

I. INTRODUCTION

Radar supports object tracking in adverse weather and provides relative velocity, but predefined motion models struggle with nonlinear road motion. PR-IMM combines a Transformer predictor with CV, CA, and CT modes to draw on data-driven and physics-based priors.

I. INTRODUCTION

  • Radar maintains robust detection in rain and fog and directly measures relative velocity through the Doppler effect.The paper motivates radar as a sensor for safe-driving tracking and decisions.Source: §I · p. 1

A. Kalman-Filter-Based Object Tracking

  • CV, CA, and CT each encode a particular motion pattern, while IMM combines their priors by mode probabilities but remains limited by predefined patterns.Those assumptions constrain representation of nonlinear motion in real road environments.Source: §I · p. 1

II. RELATED WORKS

  • Transformer predictors can learn nonlinear motion from past trajectories, but their predictions can become unstable under large observation noise or out-of-distribution inputs.The cited related work describes self-attention as modeling long-range time-series dependencies.Source: §I · p. 1
  • Motion-model and data-driven approaches offer complementary strengths: learned models capture diverse patterns, while Kalman filtering handles measurement uncertainty through covariance.The paper presents combining these strengths as a promising direction for object tracking.Source: §I · p. 1

I. INTRODUCTION

  • PR-IMM combines a Transformer predictor with CV, CA, and CT as IMM modes, probabilistically weighting their priors to use suitable information under different motion conditions.The paper evaluates the method on RadarScenes and reports improvements over single models and conventional IMM.Source: §I · p. 1

B. IMM-Based Object Tracking

  • IMM handles a broader range of motion than one model, yet maneuvers outside all predefined models can leave every mode approximating the true motion poorly.The resulting weighted combination can increase state-estimation error.Source: B. IMM-Based Object Tracking · p. 2

C. Deep-Learning-Based Object Tracking

Neural predictors can supply future positions for data association, but inaccurate predictions may lack an uncertainty-based correction. PR-IMM combines learned and physics-based priors, beginning with radar measurements and Kalman prediction.

C. Deep-Learning-Based Object Tracking

  • Neural predictors estimate future positions for data association, but substantial prediction errors can accumulate without an intrinsic uncertainty-based correction.The cited work notes that Transformer predictors can capture longer-range temporal dependencies than recurrent models.Source: C. Deep-Learning-Based Object Tracking · p. 2

D. Existing Combinations of Kalman Filtering and Deep Learning

  • In prior Kalman-filter and deep-learning combinations, learning is mainly auxiliary, with no independent alternative mode available when its prediction is inaccurate.The cited approaches include a neural network coupled to the state model and LSTM-based error compensation.Source: D. Existing Combinations of Kalman… · p. 2

III. PR-IMM-BASED MULTIPLE OBJECT TRACKING

  • PR-IMM integrates a Transformer that learns nonlinear motion with CV, CA, and CT priors that can compensate when the learned prediction is unreliable.The method aims for robust position estimation under noisy observations.Source: §III · p. 2

A. Overall System Pipeline

  • The tracking pipeline assumes detection is complete and averages radar points sharing a RadarScenes track ID into an object-level measurement.The measurement includes longitudinal and lateral position, radial velocity, and RCS.Source: A. Overall System Pipeline · p. 2

B. Kalman-Filter Prediction with Motion Models

  • CV, CA, and CT use model-specific state vectors and transition functions to predict states and covariances from previous posteriors.CV and CA use linear transitions; CT uses a nonlinear transition and Jacobian-based covariance propagation.Source: B. Kalman-Filter Prediction with… · pp. 2–3

1) CV Model:

The CV and CA models encode different assumptions about object motion and use radar velocity or measurement correction to initialize or refine their states.

1) CV Model:

  • CV assumes straight-line motion at constant velocity and represents longitudinal and lateral positions and velocities.Its initial velocity components come from measured radar radial velocity and are subsequently refined by Kalman correction.Source: 1) CV Model · p. 3

2) CA Model:

  • CA assumes constant acceleration and augments the CV state with acceleration components.Initial accelerations are set to zero and later corrected using measurements.Source: 2) CA Model · p. 3

3) CT Model:

The CT model handles turning with a nonlinear transition, while the Transformer-based PR model predicts next-step displacement from recent radar observations. Its architecture encodes temporal order and uses separate learned queries for x- and y-displacement.

3) CT Model:

  • CT initializes turn rate at zero and refines it through Kalman correction, using a nonlinear transition when turn rate is nonzero.Near-zero turn rate uses a Taylor-series approximation to avoid numerical divergence, with covariance propagated through the transition Jacobian.Source: 3) CT Model · p. 3

C. Transformer-Based Prediction

  • PR predicts next-step object displacement from up to five prior radar observations using relative position, radial velocity, RCS, and relative time.Historical positions are relative to the latest observation so the model learns motion patterns rather than absolute scene coordinates.Source: C. Transformer-Based Prediction · p. 3, 1) PR Model Architecture · p. 3

1) PR Model Architecture:

  • PR adds positional embeddings to encode temporal order and appends two learnable queries for x- and y-direction displacement prediction.The Transformer encoder outputs the two displacements, which are added to the previous posterior PR position to form the prior position.Source: 1) PR Model Architecture · p. 3, 2) PR Filter · p. 3

2) PR Filter:

PR-IMM mixes shared position information across its four modes before prediction, while preserving each mode’s distinct non-position state. The resulting fusion uses projected positions and conservatively accounts for disagreement among modes.

D. Prior State Fusion

  • PR-IMM projects each mode’s prior position through its observation matrix and uses those positions for state fusion despite differing full-state dimensions.Source: D. Prior State Fusion · p. 4

1) Input Mixing:

  • Before prediction, each destination mode combines the previous posterior positions and position covariances using prior mode and transition probabilities, supporting continuity when the dominant mode changes.Source: 1) Input Mixing · p. 4
  • The mixed position covariance includes dispersion from disagreement among modes, while input mixing is limited to common position and position covariance because full states differ in dimension and structure.Source: 1) Input Mixing · p. 4
  • Each mode retains its own non-position state variables when forming the mixed full state.Source: 1) Input Mixing · p. 4

2) PR-IMM Prior State Fusion:

PR-IMM combines predicted positions according to current prior mode probabilities. The fused prior position is then used in track-observation matching.

E. Data Association

Data association uses the PR-IMM prior position and a radial-velocity difference to score track-observation matches. A Hungarian-algorithm assignment pairs tracks and observations within the same object class.

  • The association cost combines Euclidean distance between the PR-IMM prior position and observation position with their radial-velocity difference.Source: E. Data Association · p. 4
  • The radial-velocity term distinguishes spatially close objects moving at different velocities, is capped at 5 m/s, and is weighted by λv = 0.1.Source: E. Data Association · p. 4
  • A Hungarian algorithm performs global one-to-one assignment for tracks and observations of the same object class; unassigned observations initialize new tracks.Source: E. Data Association · pp. 4–5

F. Posterior State Fusion

After association, each mode uses its prediction-observation innovation for correction and mode-probability updates. The final correction uses full-state covariance, allowing position innovations to update non-position states through cross-covariances.

1) Mode Probability Update:

  • Each mode computes an innovation and innovation covariance after an observation is assigned; these support Kalman correction and posterior mode-probability updates.Source: 1) Mode Probability Update · p. 5
  • For the PR mode, QPR is set smaller than RPR so the learned prior state is treated as relatively reliable compared with the raw observation.Source: 1) Mode Probability Update · p. 5

2) Correction:

  • Although observations contain only position, each mode’s full prior covariance lets the position innovation update velocity, acceleration, and turn-rate states through cross-covariances.Source: 2) Correction · p. 5

3) PR-IMM Posterior State Fusion:

PR-IMM combines the four modes’ position estimates according to their posterior mode probabilities. The experiments use a sequence-based RadarScenes split and compare six tracking methods.

3) PR-IMM Posterior State Fusion:

  • The PR, CV, CA, and CT position estimates are dynamically combined according to the motion situation at each time step.The final position weights each mode’s posterior position by its posterior mode probability.Source: 3) PR-IMM Posterior State Fusion · p. 5

1) Dataset:

  • RadarScenes’ 158 sequences are split into 6 validation, 22 evaluation, and 130 training sequences.The split is by sequence.Source: 1) Dataset · p. 5, Table I · p. 5
  • The six compared methods are CV, CA, and CT Kalman filters, conventional IMM, standalone PR, and the proposed hybrid.The standalone PR provides a data-driven comparison to the physics-based baselines and hybrid framework.Source: 2) Compared Methods · p. 5

3) Evaluation Metrics:

The section defines position and identity-preservation metrics, then compares PR-IMM’s prior prediction and tracking performance across overall and scenario-wise evaluations.

3) Evaluation Metrics:

  • Position prediction is evaluated with RMSE; identity preservation uses IDSW, which increments when an object's estimated track ID changes, and IDF1.Lower IDSW and higher IDF1 indicate better identity-preservation performance.Source: 3) Evaluation Metrics · pp. 5–6

B. Quantitative Performance Comparison

C. Qualitative Analysis

PR-IMM combines a Transformer prediction mode with CV, CA, and CT, and the supplied analyses report best comparative performance and stable trajectory agreement with ground truth. Mode probabilities show that PR is not dominant alone, consistent with complementary physics-based modes.

2) Object Trajectory Visualization:

1) Mode Probability Visualization:

V. CONCLUSION

  • PR-IMM integrates the Transformer predictor as an IMM mode with CV, CA, and CT, using shared 2D position and covariance to mix inputs and fuse states across differing state dimensions.Source: §V · p. 7
  • PR-IMM achieves the best performance among CV, CA, and CT single-motion models, the conventional IMM, and the standalone PR model.Source: §V · p. 7

Related papers

Loading 2609.13307v1…