Source-linked AI summary
Deep OC-SORT: Multi-Pedestrian Tracking by Adaptive Re-Identification
Gerard Maggiolino, Adnan Ahmad, Jinkun Cao, Kris Kitani
TL;DR
The paper addresses the limited robustness of simple appearance heuristics for integrating visual cues with motion-based multi-object tracking. It introduces adaptive re-identification on top of OC-SORT, combining weighted appearance similarity with motion association and camera motion compensation. Deep OC-SORT achieves leading benchmark performance across MOT17, MOT20, and DanceTrack.
Problem
Integrating visual appearance robustly with high-performing motion-based multi-object tracking remains under-explored beyond simple heuristic models.
Method
Deep OC-SORT adaptively combines weighted appearance similarity with OC-SORT's motion-based association and adds camera motion compensation.
Results
Deep OC-SORT achieves leading published-method performance across MOT17, MOT20, and DanceTrack, including 63.9 HOTA on MOT20-test.
Takeaways & Limitations
The method provides a strong baseline for future multi-object tracking studies using both motion and appearance cues.
Abstract
from arXiv · showhide
Motion-based association for Multi-Object Tracking (MOT) has recently re-achieved prominence with the rise of powerful object detectors. Despite this, little work has been done to incorporate appearance cues beyond simple heuristic models that lack robustness to feature degradation. In this paper, we propose a novel way to leverage objects' appearances to adaptively integrate appearance matching into existing high-performance motion-based methods. Building upon the pure motion-based method OC-SORT, we achieve 1st place on MOT20 and 2nd place on MOT17 with 63.9 and 64.9 HOTA, respectively. We also achieve 61.3 HOTA on the challenging DanceTrack benchmark as a new state-of-the-art even compared to more heavily-designed methods. The code and models are available at \url{https://github.com/GerardMaggiolino/Deep-OC-SORT}.
1. INTRODUCTION
Deep OC-SORT addresses the under-explored integration of appearance with motion-based matching by adaptively incorporating visual cues into OC-SORT. The method is designed to improve robustness despite noisy appearance features and adds camera motion compensation.
- Contributions: Deep OC-SORT adaptively integrates visual appearance with motion-based association in a single stage.It builds on the pure motion-based OC-SORT algorithm rather than directly adapting DeepSORT's approach to SORT.
- Motivation: Bounding-box appearance features remain noisy under occlusion, motion blur, and similar object appearances.The proposed approach uses visual comparison without requiring fine-grained semantics such as instance segmentation.
- Contributions: Camera motion compensation complements the object-centric motion model in moving scenes.The method introduces this component alongside adaptive appearance integration.
- Novelty: The adaptive appearance mechanism is newly designed rather than a straightforward DeepSORT adaptation of SORT.The paper names the resulting method Deep OC-SORT.
2. RELATED WORKS
Motion-based multi-object tracking remains competitive despite advances in object detectors. This line of work centers on Kalman-filter-based tracking-by-detection, with methods such as SORT and ByteTrack addressing association and missing predictions.
- Kalman filtering underpins a prominent family of tracking-by-detection methods.
- SORT associates tracks using a linear motion assumption and intersection over union.
- ByteTrack addresses missing predictions by incorporating low-confidence candidates during association.
3. METHODS
Deep OC-SORT extends OC-SORT with camera motion compensation, dynamic appearance updates, and adaptive appearance weighting. These modules correct motion states, filter degraded embeddings, and emphasize discriminative track-detection matches.
- Method overview: Deep OC-SORT consists of Camera Motion Compensation, Dynamic Appearance, and Adaptive Weighting modules built on OC-SORT.The algorithm pipeline is illustrated in Figure 1 and retains OC-SORT’s overall pipeline and Hungarian matching.
- Camera Motion Compensation: Camera Motion Compensation applies a scaled rotation and translation to OC-SORT states and bounding-box components before Kalman extrapolation.The corrected states provide the basis for prediction in moving scenes, where OC-SORT depends strongly on detection quality.
- Dynamic Appearance: Dynamic Appearance replaces the fixed EMA coefficient with detector-confidence-dependent α_t, selectively incorporating appearance information in high-quality detections.At the confidence threshold, α_t = 1 and the new embedding is ignored; at confidence 1, α_t = α_f and the new embedding is maximally incorporated.
- Dynamic Appearance: Dynamic Appearance uses low detector confidence as a proxy for occlusion or blur, allowing corrupted embeddings to be rejected without adding hyper-parameters.The fixed EMA value is α_f = 0.95, and the dynamic operation preserves the standard EMA structure.
- Adaptive Weighting: Adaptive Weighting increases appearance weight for track-box pairs whose appearance similarity is discriminative within a matrix row or column.Discriminativeness is measured by the difference between the highest and second-highest appearance scores, while ε caps the boost for large gaps.
- Adaptive Weighting: The final association cost combines IoU cost with appearance cost weighted by the global appearance weight plus the pair-specific boost.The method uses the first- and second-highest scores because lower-scoring matches are considered irrelevant to a true positive appearance match.
4. EXPERIMENTS
Experiments evaluate Deep OC-SORT on MOT17, MOT20, and DanceTrack using HOTA and ablations of its modules. The method achieves strong benchmark rankings, while ablations show consistent benefits from appearance and adaptive weighting.
- Benchmark Results: 64.9 HOTA ranks Deep OC-SORT 2nd on MOT17-test, while 63.9 HOTA ranks it 1st on MOT20-test.Using the same detections, it outperforms the cited existing comparisons.
- Benchmark Results: Deep OC-SORT achieves a new state-of-the-art among published methods on the challenging DanceTrack benchmark.DanceTrack features heavy occlusion and frequent crossovers.
- Ablation Study: Ablations evaluate Appearance Embedding, Dynamic Appearance, Camera Motion Compensation, and Adaptive Weighting against an OC-SORT baseline.The study uses validation sets from MOT17, MOT20, and DanceTrack.
- Ablation Study: Appearance cues improve performance across all datasets and metrics, while Dynamic Appearance adds further gains with negligible computation and no additional hyper-parameters.These comparisons are made against the OC-SORT baseline and preceding module additions.
- Ablation Study: Camera Motion Compensation improves MOT17-val and DanceTrack-val but provides no improvement on static-camera MOT20-val.The result links CMC’s benefit to the camera setting described for MOT20-val.
5. CONCLUSION
Deep OC-SORT extends motion-only OC-SORT by adaptively incorporating visual appearance through weighted appearance similarity and blended visual costs. Camera motion compensation additionally supports tracking under moving cameras.
- 5. CONCLUSION: Deep OC-SORT uses weighted appearance similarity across detection-track matches to create a blended visual cost.This is the paper’s adaptive re-identification mechanism for adding appearance to OC-SORT.
- 5. CONCLUSION: Camera Motion Compensation provides benefits for tracking objects under moving cameras.The conclusion presents CMC as an additional component alongside adaptive appearance integration.