Source-linked AI summary
Hard to Track Objects with Irregular Motions and Similar Appearances? Make It Easier by Buffering the Matching Space
Fan Yang, Shigeyuki Odashima, Shoichi Masui, Shan Jiang
TL;DR
MOT methods can fail when targets have indistinguishable appearances and irregular motions, making appearance features unreliable and geometric matching ambiguous. C-BIoU expands the matching space with proportional buffers and applies cascaded matching to control overexpansion. It improves HOTA across challenging MOT datasets and serves as the dominant component of second-place SoccerNet MOT and MOTComplex DanceTrack solutions.
Problem
Indistinguishable appearances and irregular motions make dependable multi-object tracking difficult for conventional MOT methods.
Method
C-BIoU uses proportional buffers to match non-overlapping detections and tracks, then applies small-buffer and large-buffer cascaded matching.
Results
2.6 to 7.2 HOTA points: C-BIoU improves tracking across MOT datasets focused on irregular motions and indistinguishable appearances.
Takeaways & Limitations
C-BIoU is a practical tracker for challenging MOT settings, combining strong tracking performance with nearly doubled speed versus OC-SORT.
Takeaways & Limitations
C-BIoU may not be robust to extremely noisy detections, and at a 40% noise ratio it can perform worse than normal IoU tracking.
Abstract
from arXiv · showhide
We propose a Cascaded Buffered IoU (C-BIoU) tracker to track multiple objects that have irregular motions and indistinguishable appearances. When appearance features are unreliable and geometric features are confused by irregular motions, applying conventional Multiple Object Tracking (MOT) methods may generate unsatisfactory results. To address this issue, our C-BIoU tracker adds buffers to expand the matching space of detections and tracks, which mitigates the effect of irregular motions in two aspects: one is to directly match identical but non-overlapping detections and tracks in adjacent frames, and the other is to compensate for the motion estimation bias in the matching space. In addition, to reduce the risk of overexpansion of the matching space, cascaded matching is employed: first matching alive tracks and detections with a small buffer, and then matching unmatched tracks and detections with a large buffer. Despite its simplicity, our C-BIoU tracker works surprisingly well and achieves state-of-the-art results on MOT datasets that focus on irregular motions and indistinguishable appearances. Moreover, the C-BIoU tracker is the dominant component for our 2-nd place solution in the CVPR'22 SoccerNet MOT and ECCV'22 MOTComplex DanceTrack challenges. Finally, we analyze the limitation of our C-BIoU tracker in ablation studies and discuss its application scope.
I. INTRODUCTION
MOT becomes difficult when objects have indistinguishable appearances and irregular motions, because conventional appearance and geometric matching can fail. The proposed C-BIoU tracker expands the matching space with buffers and uses cascaded matching to improve association in these conditions.
- Motivation: Irregular motions and indistinguishable appearances challenge conventional MOT methods, causing performance drops on datasets such as DanceTrack.Methods that perform well on MOT17 may fail when objects move irregularly and look similar.
- Buffered IoU: BIoU expands the matching space without changing bounding-box centers, scale ratios, or shapes.The buffer is proportional to the original bounding box.
- Buffered IoU: BIoU directly matches identical objects whose detections and tracks do not overlap in adjacent frames.This addresses mismatches caused by fast movement or inaccurate motion estimation.
- Cascaded Matching: Cascaded matching first uses a small buffer for alive tracks and detections, then a large buffer for unmatched pairs.The cascade reduces the risk of overexpanding the matching space.
- Results: 2.6 to 7.2 HOTA points: C-BIoU improves tracking across MOT datasets focused on irregular motions and indistinguishable appearances.The tracker was also the dominant component of second-place solutions in the SoccerNet MOT and MOTComplex DanceTrack challenges.
II. RELATED WORKS
MOT association relies on appearance and geometric consistency, but similar-looking targets and irregular motion weaken both assumptions. Existing geometric matching based on IoU can fail when bounding boxes do not overlap or motion estimates are inaccurate.
- Appearance Consistency: Appearance consistency is useful for cross-frame association, but it becomes unreliable when tracking targets have similar appearances.DanceTrack, SoccerNet, and GMOT-40 expose this challenge.
- Geometric Consistency: IoU is commonly used to measure geometric consistency between detections and track predictions.Motion estimation commonly uses Kalman filters, while other approaches use neural networks or Bayesian filters.
- Geometric Consistency: Irregular motions by soccer players and dancers can cause motion-estimation models to fail.Non-stationary cameras add further difficulty because image registration is time-consuming and its accuracy is not guaranteed.
B. Geometric Consistency Measurement
The tracker replaces ordinary IoU matching with BIoU to handle non-overlapping boxes and motion-estimation bias. It combines this measurement with cascaded matching to limit overexpansion of the matching space.
- Geometric Consistency Measurement: IoU assigns zero similarity to all non-overlapping boxes, which can break initialization and ongoing tracking under irregular motion.BIoU expands the matching space to measure geometric consistency despite fast motion and estimation bias.
- Cascaded Matching: C-BIoU first matches alive tracks and detections with a small buffer, then matches remaining pairs with a large buffer.This ordering follows the principle of matching confident samples before ambiguous ones.
- Tracker Design: The tracker inherits part of SORT's track management and uses C-BIoU for geometric consistency measurement.Its design targets objects with indistinguishable appearances and irregular motions.
A. Tracking Pipeline
C-BIoU follows a tracking-by-detection pipeline: an external detector supplies framewise boxes, and the tracker associates them into trajectories. It uses buffered matching and recent-frame average motion to respond simply and quickly to unpredictable changes.
- Tracking Pipeline: The pipeline applies an off-the-shelf detector to generate bounding boxes, which C-BIoU then uses to produce tracking results.Separating detection from tracking allows the tracker to operate on arbitrary detections.
- Buffered IoU: BIoU buffers are proportional to detections and tracks while preserving their centers, scale ratios, and shapes.Two buffer scales are selected by grid search for cascaded matching.
- Simple Motion Estimation: C-BIoU averages motions from recent matched detections instead of using a Kalman filter for state estimation.The default number of recent frames is 2 ≤ n ≤ 5.
- Simple Motion Estimation: The simple motion-estimation approach increases tracking speed by reducing computation associated with Kalman-filter calculations.The buffered track state is used for data association at later frames.
D. Track Management
The tracker manages tracks through initialization, matching, updating, and termination. Its cascaded matching first uses a small buffer, then applies a large buffer to remaining tracks and detections.
- New tracks initially use their recorded bounding boxes as states because motion cannot yet be estimated reliably.
- Tracks are initialized from unmatched detections and terminated after remaining unmatched for a specified max age.
- The first association matches alive tracks and detections using BIoU with the small buffer scale b1 and linear assignment.
- A second association applies BIoU with the large buffer scale b2 to unmatched tracks and detections affected by large irregular motions.
- The prototype can be combined with other MOT modules to construct a more powerful tracking framework.
IV. EXPERIMENTS
The experiments evaluate C-BIoU across four MOT datasets using complementary tracking metrics and analyze its modules, buffer scales, and sensitivity to detection noise. The evaluation includes both conventional and irregular-motion scenarios.
- Experimental Design: The experiments compare C-BIoU with state-of-the-art methods on four MOT datasets using official or prior-study results where applicable.
- Ablation Studies: Ablation studies examine BIoU, cascaded matching, motion modeling, buffer scales, detection noise, and corresponding limitations.
- Datasets: Four datasets cover conventional MOT17 scenes and irregular-motion, similar-appearance scenarios in DanceTrack, SoccerNet, and GMOT-40.
- Evaluation Metrics: HOTA, DetA, AssA, MOTA, and IDF1 evaluate tracking from detection, association, overall MOT, and identity perspectives.
- Evaluation Metrics: Tracking inference speed is reported in FPS without the detection component, using an Intel Xeon Silver 4216 CPU.
B. Main Results
C-BIoU performs similarly to competing methods on conventional MOT17 while delivering its largest gains on DanceTrack and irregular-motion tracking benchmarks. It also provides substantially faster tracking inference than several alternatives.
- On MOT17, C-BIoU achieves a similar HOTA score to other methods, while DanceTrack exposes larger tracking challenges.
- 4.9 HOTA points separate C-BIoU from OC-SORT on DanceTrack, establishing a new state of the art in that comparison.
- Almost 2× the tracking inference speed of OC-SORT is achieved on MOT17 and DanceTrack without the detection component.
2) Comparisons Using Oracle Detections:
Using oracle detections isolates association quality from detection errors. C-BIoU produces fewer tracking errors than SORT and OC-SORT, while its oracle-detection results remain imperfect.
- 2.6 to 7.2 HOTA points of improvement are reported over other methods across DanceTrack, SoccerNet, and GMOT-40 with oracle detections.
- Oracle detections let the experiments focus on tracking performance rather than detection quality.
- Even with oracle detections, the reported tracking results are still imperfect.
- C-BIoU generates fewer tracking errors than SORT and OC-SORT in examples from DanceTrack and SoccerNet.
C. Ablation Experiments
The ablations show that BIoU, cascaded matching, and motion estimation each contribute to C-BIoU performance, with cascade matching especially valuable under fast camera-induced motion.
- Effect of the BIoU: BIoU achieves higher HOTA than IoU, GIoU, and DIoU within the same tracker framework.The comparison removes cascaded matching and motion estimation to isolate geometric matching.
- Effect of Cascaded Matching: A HOTA gain of 1.2 results from integrating cascaded matching and BIoU on SoccerNet.The non-stationary camera produces extremely fast object motion, making cascade matching more robust in this setting.
- Effect of Motion Estimation: Motion estimation improves HOTA when combined with BIoU, even when implemented by averaging previous motions.BIoU compensates the matching space for incorrect motion estimation.
2) Effect of Buffer Scales in the C-BIoU Tracker:
Buffer-scale selection substantially affects C-BIoU, while detection noise defines an important robustness boundary: moderate noise is tolerated, but extreme noise can reverse the advantage over IoU.
- Effect of Buffer Scales: The [0.3, 0.4] buffer-scale combination gives the maximum HOTA score on the DanceTrack validation set.The study evaluates 21 combinations with b1 < b2, where both scales range from 0.1 to 0.7.
- Effect of Detection Noise: The authors evaluate detection-quality effects by injecting matched-ratio false negatives and false positives into oracle DanceTrack detections.Detections are first removed to create false negatives, then added at non-target locations to create false positives.
- Effect of Detection Noise: At a 40% noise ratio, C-BIoU can perform worse than the normal IoU tracker.High-ratio noisy detections leave tracks and detections non-overlapping for longer intervals and make buffered matching more aggressive toward false positives.
- Effect of Detection Noise: At noise ratios no higher than 20%, C-BIoU maintains the best performance.The buffer helps match objects whose track and detection briefly fail to overlap.
V. CONCLUSION AND LIMITATION DISCUSSION
The paper concludes that buffered matching helps C-BIoU handle irregular motion and indistinguishable appearances, while extreme detection noise remains its main limitation.
- Conclusion: C-BIoU expands the matching space to match identical non-overlapping detections and tracks and compensate for motion-estimation bias.These two mechanisms are presented as the basis for mitigating irregular-motion effects.
- Conclusion: C-BIoU outperforms most existing methods on MOT datasets featuring indistinguishable appearances and irregular motions.The authors describe the results as suggesting generalizability within this application setting.
- Limitation: C-BIoU may not be robust to extremely noisy detections.The conclusion also notes simplicity, fast speed, and good tracking performance as reasons it remains applicable to real-world settings.