Source-linked AI summary
Online Multi-Object Tracking with Instance-Aware Tracker and Dynamic Model Refreshment
Peng Chu, Heng Fan, Chiu C Tan, Haibin Ling
TL;DR
SOT trackers can confuse similar targets in MOT, and detector-based initialization may be noisy. The paper proposes instance-aware target models, exclusive assignment, and learned dynamic refreshment. It reports best overall performance against published results on MOT15 and MOT16.
Problem
SOT trackers may confuse similar targets in MOT, while detector-provided target candidates can have noisy locations and scales.
Source: §1 · p. 1Method
The method combines target-versus-background and target-versus-instance information, assigns locations exclusively across target models, and uses learned dynamic model refreshment.
Source: §1 · pp. 1–2Results
Takeaways & Limitations
The authors conclude that instance-awareness within and between target models supports online location prediction while preserving model uniqueness and preventing duplicated or false-positive trajectories.
Source: §6 · p. 8Each spatial candidate can be assigned to at most one tracked target, and model parameters are determined using images and target locations only up to the previous frame.
Source: §4.1 · p. 3
Abstract
from arXiv · showhide
Recent progresses in model-free single object tracking (SOT) algorithms have largely inspired applying SOT to \emph{multi-object tracking} (MOT) to improve the robustness as well as relieving dependency on external detector. However, SOT algorithms are generally designed for distinguishing a target from its environment, and hence meet problems when a target is spatially mixed with similar objects as observed frequently in MOT. To address this issue, in this paper we propose an instance-aware tracker to integrate SOT techniques for MOT by encoding awareness both within and between target models. In particular, we construct each target model by fusing information for distinguishing target both from background and other instances (tracking targets). To conserve uniqueness of all target models, our instance-aware tracker considers response maps from all target models and assigns spatial locations exclusively to optimize the overall accuracy. Another contribution we make is a dynamic model refreshing strategy learned by a convolutional neural network. This strategy helps to eliminate initialization noise as well as to adapt to the variation of target size and appearance. To show the effectiveness of the proposed approach, it is evaluated on the popular MOT15 and MOT16 challenge benchmarks. On both benchmarks, our approach achieves the best overall performances in comparison with published results.
1. Introduction
The paper integrates SOT into MOT with target models that distinguish each target from both background and other instances, addressing confusion among similar targets and noisy detector initialization. It also introduces learned model refreshment, and reports the best overall performance against published results on MOT15 and MOT16.
1. Introduction
- Similar targets can confuse multiple SOT trackers, causing drift or duplicate tracking, while noisy detector locations and scales challenge target initialization.The paper also notes that tracking-by-detection separates detection from frame-to-frame correlation and depends on detector reliability.Source: §1 · p. 1
- Instance-aware models combine target-versus-background and target-versus-instance information, while global response-map optimization assigns each target exclusively to one model.Detection verification incorporates detector detections and target-model predictions to solve the global optimization problem efficiently.Source: §1 · pp. 1–2
- The authors identify mutual instance awareness as a way to improve the handling of ambiguity among nearby similar targets.The tracker integrates SOT techniques into MOT while modeling awareness within and between targets.Source: §1 · pp. 1–2
- A learned adaptive refreshment strategy is proposed to improve SOT reliability in MOT.The refreshment mechanism is learned via a convolutional neural network.Source: §1 · p. 2
- The approach achieves the best overall performance against published results on both MOT15 and MOT16.The authors identify the two challenge benchmarks as the evaluation setting.Source: §1 · p. 2
2. Related Work
The related work contrasts detection-dependent tracking-by-detection methods with online approaches and the paper’s use of per-target tracking models. Its stated distinction from other online methods is awareness between SOT trackers and dynamic model refreshment.
2. Related Work
- Offline global-optimization methods use past and future observations and data association, and their performance depends heavily on external detector quality.The cited approaches include Hungarian algorithm [6,19], network flow, and multiple hypotheses tracking.Source: §2 · p. 2
- Unlike tracking-by-detection approaches, this method learns an online model for each target; detections verify model uniqueness and support refreshment.The paragraph contrasts this approach with methods that associate detections into trajectories.Source: §2 · p. 2
- Compared with other online methods, the paper adds awareness between SOT trackers and dynamic model refreshment to address initialization noise.Examples of online approaches discussed include learned policies, deep CNN and LSTM [42], spatial-temporal attention [10], and detector/SOT candidate selection.Source: §2 · p. 2
3. System Overview
For each frame, the system uses image data and external detections alongside target-model predictions. Verification assigns candidates exclusively, supports model refreshment and recovery, and routes remaining unmatched items to target management.
3. System Overview
- Each frame’s image and external detections are input to the system, while target models predict locations and score detections independently.Source: §3 · p. 2
- Detection verification assigns each spatial candidate to at most one target and refreshes a verified target model when its detection encloses the target better than the model prediction.Source: §3 · p. 2
- Backup-model matching can recover from incorrect refreshment; remaining unpaired predictions and detections enter occlusion handling, with final unverified targets exiting and unpaired detections added as new targets.Source: §3 · p. 3
4. Methodology
The method formulates online MOT as an assignment problem, then combines target-specific instance awareness with detection verification and adaptive model refreshment. Target management also handles occlusion recovery, new entries, and exits.
4.1. Problem Formulation
- The online formulation selects candidate locations for tracked targets to maximize overall tracking quality, with each candidate assigned to at most one target and model parameters learned from prior frames.The objective is solved frame by frame.Source: §4.1 · p. 3
4.2. Instance-Aware Tracker
- Instance-aware tracking combines target-versus-background and target-versus-instance discrimination, then globally assigns locations using responses from all target models under spatial exclusivity.The target-specific objective separates detection from background and instance identity; the global optimization prevents multiple targets from claiming the same image location.Source: §4.2 · pp. 3–4
4.3. Detection Verification
- Detection verification combines detector outputs with tracker predictions as candidate locations, then uses graph multicut optimization to assign detections and periodically verify model uniqueness.The verification interval TV also controls dependence on external detections; increasing it lets models track independently for longer.Source: §4.3 · pp. 4–5
4.4. Model Refresh
- A CNN classifier decides whether an assigned detection should refresh a target model, comparing its bounding box with the tracker prediction; reinforcement learning updates the policy when it errs.The refresh strategy addresses noisy detection-based initialization and changing target scale; the classifier is trained on mistaken refresh decisions.Source: §4.4 · p. 5
4.5. Target Management
- Target management uses an SVM to match detections with targets after occlusion, adds unmatched detections as new targets, and checks exits by out-of-view status or prolonged lack of detection.Occlusion recovery assumes an unassigned detection is either a new target or a target that has reappeared, and uses Hungarian matching for final pairings.Source: §4.5 · pp. 5–6
5. Experiments
Experiments on MOT15 and MOT16 examine the tracker’s settings and components, then compare its test-set performance with published methods. The proposed method establishes a new state of the art in MOTA on both benchmarks.
5.1. Experiment Setting
- The test sets comprise 11 MOT15 sequences and 7 MOT16 sequences, with varied camera motion, angle, and imaging conditions; similar training sequences are provided.Real-detector detections are provided for both training and test sets. The system runs at about 0.3 fps on MOT15 and 0.1 fps on MOT16, whose average per-frame target densities are 10.6 and 30.8, respectively.Source: §5 · p. 6, §5.1 · pp. 6–7
5.2. Determine TV
- TV controls the interval between detection verification: higher values make target models act more independently and may increase FP, while lower values increase dependence on detection and may increase FN when detection quality worsens.The experiments use CLEAR MOT metrics, including MOTA, MOTP, MT, ML, and Frag.Source: §5.2 · p. 7, §5.1 · p. 7
- Testing on Venice-2 and MOT16-05 found peak MOTA at TV = 4 and TV = 5, respectively; the remaining experiments use TV = 4.Across both sequences, MOTA first increased and then decreased as TV rose, due to increasing FP.Source: §5.2 · p. 7
5.3. Ablation Study
- On the Venice-2 training sequence, disabling detection verification caused the greatest performance degradation, while model refreshment improved tracking accuracy and bounding-box precision.The full method’s interpolation for re-tracking and occlusion handling adds FP but reduces more FN, improving overall performance.Source: §5.3 · p. 8
5.4. Results on Test Sequences
- The method establishes a new state of the art in MOTA among online and offline methods on MOT15 and MOT16, with the least FN and best MT/ML among online methods.The result is reported for the benchmark test sequences; the challenge includes more than 10 times as many FN as FP on MOT16, partly due to public detections.Source: Table 3 · p. 7, Table 4 · p. 7, §5.4 · p. 8
6. Conclusion
The conclusion presents an instance-aware SOT-based approach to MOT, with dynamic model refreshing. Experiments on MOT15 and MOT16 show effectiveness compared with the state of the art.
6. Conclusion
- The approach combines SOT with instance awareness within each target model and across target models to improve online target-location prediction and preserve model uniqueness.The paper says this helps prevent duplicated and false-positive trajectories.Source: §6 · p. 8
- A learned convolutional neural network dynamically refreshes tracking models to reduce noise from inaccurate detections and adapt to target appearance and scale variation over time.Source: §6 · p. 8
- Experiments on MOT15 and MOT16 challenge datasets show the approach’s effectiveness in comparison with the state of the art.Source: §6 · p. 8