Source-linked AI summary
Simple Online and Realtime Tracking
Alex Bewley, Zongyuan Ge, Lionel Ott, Fabio Ramos, Ben Upcroft
TL;DR
Online multiple-object tracking must balance accuracy with realtime speed, while detection quality may constrain performance. This paper presents a simple frame-to-frame tracker using classical association methods and achieves best-in-class speed and accuracy.
Problem
Online multiple-object tracking faces a pronounced accuracy–speed trade-off, while detection quality may limit tracking performance.
Method
The framework combines CNN-based detection with Kalman-filter motion prediction and Hungarian-algorithm data association for frame-to-frame tracking.
Results
The tracker achieves best-in-class performance in both speed and accuracy, with the highest MOTA among online trackers and performance comparable to NOMT.
Takeaways & Limitations
Its simplicity makes the framework well suited as a baseline for research on long-term occlusion and object re-identification.
Takeaways & Limitations
Tracks are terminated after one missed detection, so object re-identification and long-term occlusion are outside the framework’s scope.
Abstract
from arXiv · showhide
This paper explores a pragmatic approach to multiple object tracking where the main focus is to associate objects efficiently for online and realtime applications. To this end, detection quality is identified as a key factor influencing tracking performance, where changing the detector can improve tracking by up to 18.9%. Despite only using a rudimentary combination of familiar techniques such as the Kalman Filter and Hungarian algorithm for the tracking components, this approach achieves an accuracy comparable to state-of-the-art online trackers. Furthermore, due to the simplicity of our tracking method, the tracker updates at a rate of 260 Hz which is over 20x faster than other state-of-the-art trackers.
1. INTRODUCTION
The paper presents a lean, online multiple-object tracker focused on efficient frame-to-frame data association for realtime applications. It prioritizes detection quality and simple, reliable tracking over complex handling of appearance, occlusion, and detection errors.
- Motivation: The proposed framework performs online multiple-object tracking by associating bounding-box detections between the previous and current frames.The tracker is designed for online operation rather than batch processing.
- Motivation: The paper treats MOT primarily as a data-association problem supported by motion and appearance modelling.Its design choices are motivated by observations from a recently established visual MOT benchmark.
- Design philosophy: The approach emphasizes efficient handling of common frame-to-frame associations instead of complex components for edge cases and detection errors.It exploits advances in visual object detection rather than aiming to make tracking robust to detection errors.
- Design philosophy: The tracker uses only bounding-box position and size, ignoring additional appearance features and explicit short- and long-term occlusion handling.The paper argues that re-identification complexity can add overhead that limits realtime use.
- Contributions: The main contributions are CNN-based detection for MOT, a Kalman-filter and Hungarian-algorithm tracker evaluated on a recent benchmark, and open-sourced code.The code is intended to support research experimentation and uptake in collision-avoidance applications.
2. LITERATURE REVIEW
Prior MOT methods either defer association decisions to manage uncertainty or use online appearance, motion, and bipartite matching models. A two-stage Hungarian-based method combines geometry and appearance but requires batch computation, motivating an online alternative.
- Delayed-decision MOT: MHT and JPDA defer difficult assignment decisions under high uncertainty, while later approximations and appearance-based pruning address their combinatorial complexity.These approaches remain unsuitable for online tracking because they delay decision making.
- Online association: Online trackers learn individual-object or global appearance models and often incorporate motion to associate detections with tracklets.These methods use online learning for appearance modeling, with motion providing additional association information.
- Online association: For one-to-one correspondences represented as bipartite graph matching, the Hungarian algorithm provides a globally optimal association solution.The Hungarian algorithm is used when associations are restricted to one-to-one correspondences.
- Two-stage association: Geiger et al. use two Hungarian-algorithm stages that combine geometry and appearance to form tracklets and bridge occlusion-induced trajectory breaks.The first stage associates detections across adjacent frames; the second associates tracklets to each other.
- Two-stage association: The two-step association process restricts Geiger et al.'s method to batch computation.Its tracklet formation and subsequent tracklet association prevent purely online operation.
3. METHODOLOGY
The methodology combines pedestrian detection, constant-velocity state propagation, IOU-based Hungarian assignment, and track-lifespan management for online tracking. It uses FrRCNN detections, Kalman-filter updates, and short-term track termination to support efficient frame-to-frame association.
- Overview: The method comprises detection, future-state propagation, detection-to-object association, and tracked-object lifespan management.These are the four key components described for the proposed tracker.
- Detection: FrRCNN uses a two-stage, parameter-sharing framework, and the tracker retains only person detections with output probabilities greater than 50%.The evaluated FrRCNN variants are ZF and VGG16, using default PASCAL VOC parameters.
- Detection: The best detector, FrRCNN(VGG16), produces the best tracking accuracy for both MDP and the proposed tracker.The comparison uses validation sequences and contrasts FrRCNN with ACF detections.
- Motion model: Targets follow an independent linear constant-velocity model, with associated detections updating state and velocity components solved through a Kalman filter.The state represents target-center location, scale, and bounding-box aspect ratio; the aspect ratio is constant.
- Data association: Associations minimize IOU distance between detections and predicted boxes via the Hungarian algorithm, while a minimum IOU rejects insufficient-overlap matches.IOU distance also handles short-term occlusion by favoring detections with similar scale and leaving covered targets unassigned.
- Track management: Tracks are terminated after TLost frames without detection, with TLost set to 1 to limit prediction errors and focus on frame-to-frame tracking.Object re-identification is outside the scope of the method.
4. EXPERIMENTS
The experiments evaluate SORT on MOT benchmark sequences using standard tracking metrics and compare it with online, batch, and near-online baselines. SORT achieves the highest MOTA among online trackers while highlighting the importance of runtime for realtime applications.
- Evaluation: The experiments use MOT benchmark sequences from moving and static cameras, with parameters tuned using the training/validation split reported in.The detector is FrRCNN(VGG16).
- Metrics: The evaluation combines metrics from with standard MOT metrics, including false alarms, mostly tracked and lost trajectories, false and missed detections, ID switches, and fragmentations.The metric definitions distinguish tracking quality, detection errors, identity switches, and track interruptions.
- Evaluation: SORT is evaluated on 11 MOT benchmark test sequences with withheld ground truth.The evaluation uses the MOT benchmark test server.
- Accuracy comparison: SORT achieves the highest MOTA score among the listed online trackers and is comparable to the more complex near-online tracker NOMT.The comparison includes TDAM, MDP, DP-NMS, NOMT, TBD, ALExTRAC, and SMOT.
- Speed and accuracy: Realtime tracking is essential for robotics and autonomous vehicles, whereas higher accuracy commonly coincides with slower runtime among MOT trackers.The paper presents tracker speed and accuracy jointly on the MOT benchmark.
5. CONCLUSION
The paper presents a simple online tracking framework centered on frame-to-frame prediction and association. It achieves state-of-the-art tracking quality using classical tracking methods while delivering best-in-class speed and accuracy.
- 5. CONCLUSION: The framework uses frame-to-frame prediction and association for simple online tracking.Its tracking components rely on classical methods.
- 5. CONCLUSION: Tracking quality is highly dependent on detection performance.The framework benefits from recent advances in detection to achieve state-of-the-art tracking quality.
- 5. CONCLUSION: The framework achieves best in class performance in both speed and accuracy.Other methods typically sacrifice one of these dimensions.