PaperlayerView exact revision on arXiv

Source-linked AI summary

Real-Time MDNet

Ilchae Jung, Jeany Son, Mooyeol Baek, Bohyung Han

arXiv:1808.08834v1cs.CV

TL;DR

Accurate CNN-based tracking is often too slow, and target-background discrimination alone struggles with similar object instances. The paper integrates improved RoIAlign and joint classification-embedding training into MDNet, achieving approximately 25 times speed-up with almost identical accuracy and strong benchmark performance.

  • Problem

    CNN-based trackers often struggle to combine practical speed with high accuracy, while target-background separation does not reliably distinguish similar target instances across domains.

  • Method

    The method integrates improved RoIAlign, high-resolution representations, enlarged receptive fields, and a joint binary-classification and instance-embedding loss into MDNet.

  • Results

    25 times speed-up over MDNet was achieved with almost identical accuracy, alongside outstanding performance across multiple benchmarks and real-time trackers.

  • Takeaways & Limitations

    The integrated representation and loss design supports real-time tracking while maintaining competitive accuracy without dataset-specific parameter tuning.

Abstract

from arXiv · show

We present a fast and accurate visual tracking algorithm based on the multi-domain convolutional neural network (MDNet). The proposed approach accelerates feature extraction procedure and learns more discriminative models for instance classification; it enhances representation quality of target and background by maintaining a high resolution feature map with a large receptive field per activation. We also introduce a novel loss term to differentiate foreground instances across multiple domains and learn a more discriminative embedding of target objects with similar semantics. The proposed techniques are integrated into the pipeline of a well known CNN-based visual tracking algorithm, MDNet. We accomplish approximately 25 times speed-up with almost identical accuracy compared to MDNet. Our algorithm is evaluated in multiple popular tracking benchmark datasets including OTB2015, UAV123, and TempleColor, and outperforms the state-of-the-art real-time tracking methods consistently even without dataset-specific parameter tuning.

1 Introduction

CNN-based trackers often struggle to combine accuracy with practical speed, while MDNet also lacks sufficiently discriminative representations for similar target instances. Real-Time MDNet addresses these limitations with faster, higher-resolution target representations and an instance embedding loss.

  • CNN-based visual trackers often achieve high accuracy but remain too slow for practical systems, with few methods balancing accuracy and speed.
  • MDNet processes candidate regions independently, creating high time and space complexity while inadequately distinguishing similar target instances across domains.
  • RoIAlign with bilinear interpolation reduces coarse quantization, but large regions can still lose useful localization cues within targets.
  • The proposed architecture uses RoIAlign, a high-resolution feature map, and enlarged activation receptive fields to improve candidate precision and semantic representation.
  • An instance embedding loss complements binary foreground/background classification to separate target instances with identical labels or similar semantics.
  • The tracker reports 25 times speed-up over MDNet with almost identical accuracy and strong performance across multiple benchmarks without dataset-specific tuning.

2 Related Work

Prior visual tracking methods improve efficiency or target-background separation, but distinguishing target instances with similar semantics remains challenging. Real-Time MDNet combines efficient region-based feature extraction with joint classification and instance discrimination.

  • CNN-based trackers commonly score candidate regions independently, while deep representations and correlation filters have improved tracking accuracy.
  • Only a few real-time trackers achieve competitive accuracy, using either efficient hand-crafted features or tracker-verifier interactions.
  • MDNet addresses cross-video label conflict through multi-domain learning and achieves state-of-the-art performance across multiple datasets.
  • Target-background saliency remains insufficient for discriminating object instances with similar semantics, motivating a loss with classification and instance-discrimination terms.
  • RoIPooling lowers feature-extraction cost by sharing a feature map, but its coarse map limits target localization effectiveness.

3 Efficient Feature Extraction and Discriminative Feature Learning

The method accelerates MDNet-style tracking by sharing convolutional feature extraction and using improved RoIAlign, while multi-task learning adds cross-domain instance discrimination to target-background classification.

  • 3.1 Network Architecture: A single forward pass constructs a shared feature map, from which adaptive RoIAlign extracts each proposal's feature for fully connected binary classification.The architecture uses three convolutional layers, adaptive RoIAlign, and three fully connected layers.
  • 3.2 Improved RoIAlign for Visual Tracking: Removing a pooling layer and using dilated convolution produce a denser, higher-resolution feature map while enlarging each activation's receptive field.The shared feature map is designed to preserve precise candidate representation and rich target semantics.
  • 3.2 Improved RoIAlign for Visual Tracking: Adaptive RoIAlign adjusts bilinear-interpolation bandwidth to RoI size, improving feature reliability for large objects.The bandwidth is proportional to the RoI width after conv3 relative to the output feature width.
  • 3.2 Improved RoIAlign for Visual Tracking: The improved RoIAlign outputs a 7 × 7 feature map followed by max pooling to produce a 3 × 3 map.The authors report that these changes significantly improve tracking performance in practice because representation errors can propagate over time.
  • 3.3 Pretraining for Discriminative Instance Embedding: The multi-task objective combines binary classification with an instance-embedding loss that separates target instances across domains.The network minimizes L = Lcls + α · Linst, with α controlling the balance between the two terms.
  • 3.3 Pretraining for Discriminative Instance Embedding: The output concatenates domain-specific binary scores, then uses separate softmax operations for target-background classification and cross-domain instance discrimination.Training processes one domain-specific minibatch per iteration, while the embedding loss increases current-domain positive scores and decreases scores from other domains.

4 Online Tracking Algorithm

The tracker follows an MDNet-like pipeline with offline pretraining, first-frame customization, online updates, and bounding-box refinement. It samples candidate regions around the previous target state and combines classification, regression, and adaptive update mechanisms.

  • Online training: At the first test frame, the tracker fine-tunes fully connected layers using labeled positive and negative samples, then replaces domain-specific branches with one sequence-specific branch.Convolutional layers remain fixed during subsequent online updates.
  • Main loop of tracking: Candidate regions are sampled from a Gaussian centered on the previous target state, and the target state is selected using the highest positive score in a three-dimensional translation-and-scale space.The scoring function evaluates samples drawn from the current frame.
  • Bounding-box regression: A linear bounding-box regressor trained from first-frame RoI features refines estimated target regions when the estimated target score exceeds 0.5.The regressor is applied from the second frame when the target state is sufficiently reliable.
  • Model updates: Long-term updates regularly incorporate accumulated samples, while short-term updates are triggered when the estimated target score falls below a reliability threshold.These complementary strategies are intended to preserve robustness and adaptiveness.
  • Online learning: Online learning uses hard-minibatch mining by selecting the 96 highest-scoring negatives from 1024 tested negative examples.Each minibatch contains 32 positive and 96 negative samples.
  • Offline pretraining: Offline pretraining uses samples from one domain per minibatch, with positive and negative examples selected by IoU thresholds.Training uses ImageNet-Vid data and an instance embedding loss across randomly selected videos.

5 Experiments

RT-MDNet is evaluated against real-time and state-of-the-art trackers on OTB2015, TempleColor, and UAV123 using standard accuracy measures and speed comparisons. It achieves strong accuracy and real-time performance, while experiments identify localization limits and the contributions of its components.

  • OTB2015: RT-MDNet significantly outperforms tested real-time trackers on OTB2015 in both accuracy measures.It is approximately 130, 25, and 8 times faster than C-COT, MDNet, and ECO, respectively.
  • OTB2015: 46 FPS average speed accompanies substantial gains over other real-time trackers in OTB2015 AUC and 20-pixel precision.Speed excluding first-frame processing is approximately 52 FPS; first-frame fine-tuning and bounding-box-regressor learning add computational cost.
  • Failure cases: Failure cases include target loss under significant occlusion, sudden large motion, out-of-plane rotation, similar appearances, and dramatic non-rigid changes.The authors associate these cases with Soccer, Biker, Coupon, and Jump sequences.
  • TempleColor: RT-MDNet surpasses all tested real-time trackers on TempleColor, substantially improving over ECO-HC.TempleColor contains 128 color videos, most overlapping with OTB2015.
  • UAV123: On UAV123, RT-MDNet outperforms state-of-the-art methods in precision, remains competitive in success rate, and beats ECO with approximately 8 times speed-up.The result is reported without dataset-specific parameter tuning on an aerial benchmark containing 123 videos and more than 110K frames.
  • Ablation study: Ablation results indicate that the multi-task loss and bounding-box regression both improve localization.The internal comparison includes versions without bounding-box regression and without both regression and instance embedding loss.

6 Conclusions

The proposed real-time tracker accelerates feature extraction and learns discriminative representations through improved RoIAlign and instance embedding. It achieves outstanding benchmark performance, while failure cases remain documented on several challenging sequences.

  • Applying instance embedding loss to MDNet improves performance.
  • Qualitative results are reported on Matrix, MotorRolling, Skiing, and Sylvester in OTB2015.
  • Failure cases are shown for Soccer, Biker, Coupon, and Jump, with magenta ground-truth and blue predicted bounding boxes.
  • The algorithm accelerates feature extraction with improved RoIAlign and uses multi-task learning to discriminate object instances across domains.
  • The method demonstrates outstanding performance compared with state-of-the-art techniques, especially real-time trackers, on public visual tracking benchmarks.
Loading 1808.08834v1…