PaperlayerView exact revision on arXiv

Source-linked AI summary

Ocean: Object-aware Anchor-free Tracking

Zhipeng Zhang, Houwen Peng, Jianlong Fu, Bing Li, Weiming Hu

arXiv:2006.10721v2cs.CV

TL;DR

Anchor-based Siamese trackers struggle to refine inaccurate boxes because regression is trained only on sufficiently overlapping anchors. Ocean replaces anchor refinement with anchor-free object regression and adds feature alignment for object-aware classification, achieving state-of-the-art performance across five benchmarks, including VOT-2018 EAO of 0.467 at 58 fps.

  • Problem

    Anchor-based trackers train regression on anchors with IoU ≥0.6, leaving them unable to amend weak predictions whose target overlap is small.

  • Method

    Ocean directly predicts target-box positions from pixels inside groundtruth boxes and aligns feature sampling with predicted boxes for object-aware classification.

  • Results

    Ocean achieves state-of-the-art performance on VOT-2018, VOT-2019, OTB-100, GOT-10k, and LaSOT; on VOT-2018 it reaches EAO 0.467 at 58 fps.

  • Takeaways & Limitations

    The anchor-free framework can rectify weak predictions and provides a real-time tracking approach whose performance is competitive across five benchmarks.

  • Takeaways & Limitations

    Training settings differ across recent trackers, complicating unified comparisons; the paper additionally compares Ocean and SiamRPN++ under the same training setting.

Abstract

from arXiv · show

Anchor-based Siamese trackers have achieved remarkable advancements in accuracy, yet the further improvement is restricted by the lagged tracking robustness. We find the underlying reason is that the regression network in anchor-based methods is only trained on the positive anchor boxes (i.e., $IoU \geq0.6$). This mechanism makes it difficult to refine the anchors whose overlap with the target objects are small. In this paper, we propose a novel object-aware anchor-free network to address this issue. First, instead of refining the reference anchor boxes, we directly predict the position and scale of target objects in an anchor-free fashion. Since each pixel in groundtruth boxes is well trained, the tracker is capable of rectifying inexact predictions of target objects during inference. Second, we introduce a feature alignment module to learn an object-aware feature from predicted bounding boxes. The object-aware feature can further contribute to the classification of target objects and background. Moreover, we present a novel tracking framework based on the anchor-free model. The experiments show that our anchor-free tracker achieves state-of-the-art performance on five benchmarks, including VOT-2018, VOT-2019, OTB-100, GOT-10k and LaSOT. The source code is available at https://github.com/researchmm/TracKit.

1 Introduction

Object tracking must localize arbitrary targets despite occlusion, deformation, and illumination changes, while real-time operation supports practical applications. Ocean addresses robustness limits in anchor-based Siamese trackers with anchor-free regression, feature alignment, and a real-time tracking framework.

  • Motivation: Real-time object tracking infers an arbitrary target’s location from its first-frame location despite occlusion, deformation, and illumination variation.It supports applications including surveillance, robotics, autonomous driving, and human-computer interaction.
  • Motivation: Anchor-based Siamese trackers train regression only on positive anchors with IoU ≥0.6, limiting refinement of weak predictions and contributing to drift.When predicted overlap falls below the trained range, the regression network has not learned to rectify the inaccurate box.
  • Proposed approach: Ocean directly regresses target positions and scales in an anchor-free manner, training distances from each groundtruth-box pixel to the box sides.This enables rectification when only a small region is identified as foreground.
  • Proposed approach: A feature alignment module samples features within predicted candidate boxes, producing object-aware features that improve foreground-background classification.The module uses a 2D spatial transformation to align sampling locations with predicted bounding boxes.
  • Results: The proposed tracker achieves state-of-the-art performance on five benchmarks, including VOT-2018 EAO of 0.467 at 58 fps.The introduction reports up to 92.2% and 12.8% relative improvements over SiamRPN and SiamRPN++, respectively.

2 Related Work

Related work covers Siamese trackers, anchor-free mechanisms, and feature alignment across tracking and detection. These lines of work motivate direct object localization and correspondence between visual features and candidate regions.

  • Siamese trackers: Siamese trackers learn similarity between a target and candidate patches, while SiamRPN adds a region proposal network with predefined anchors.SiamRPN can capture object scale changes through its anchor boxes.
  • Anchor-free mechanism: Anchor-free methods predict object locations directly rather than estimating offsets from predefined anchor boxes.The related work notes their growing use in object detection because of architectural simplicity and strong performance.
  • Feature alignment: Feature alignment matches visual features to reference regions of interest, supporting localization in detection and tracking.ROIAlign is cited as an object-detection example that aligns features with reference anchor boxes.

3 Object-aware Anchor-Free Networks

Ocean replaces anchor refinement with pixel-level regression and adds object-aware feature alignment for classification. Its jointly trained branches use predicted boxes to improve scale-adaptive target-background discrimination.

  • 3.1 Anchor-free Regression Network: Anchor-based regression is limited because it trains only on positive anchors with IoU ≥0.6, making small-overlap anchors difficult to refine.This motivates directly predicting target geometry rather than small anchor offsets.
  • 3.1 Anchor-free Regression Network: The regression branch treats every pixel inside a groundtruth box as a sample and predicts distances from that pixel to the box’s four sides.The labels are l*, t*, r*, and b*, corresponding to left, top, right, and bottom distances.
  • 3.1 Anchor-free Regression Network: Using all groundtruth-box pixels allows scale prediction even when only a small region is identified as foreground, helping rectify weak predictions during inference.The passage limits this capability to rectification to some extent.
  • 3.2 Object-aware Classification Network: The feature alignment module transforms fixed convolution sampling positions to align with each regression-predicted candidate box.Its spatial transformation adapts sampling to predicted box scale and extracts an object-aware feature.
  • 3.2 Object-aware Classification Network: Ocean combines object-aware and regular-region features for foreground-background classification, using IoU-based probabilistic labels for the former and center-based binary labels for the latter.The regression and classification branches are jointly optimized with IoU and binary cross-entropy losses.
  • 3.2 Object-aware Classification Network: Unlike FCOS’s fixed-region feature, Ocean’s object-aware feature captures global target appearance and adapts sampling regions to object scale changes.The combined features are reported as making classification more reliable.

4 Object-aware Anchor-Free Tracking

Ocean builds tracking from an offline object-aware anchor-free model with feature extraction, combination, and localization, plus an online branch that models target appearance changes.

  • Feature extraction: The offline tracker extracts exemplar and search features with a modified ResNet-50 backbone.
  • Feature combination: Depth-wise cross-correlation combines exemplar and search features, followed by parallel dilated convolutions and point-wise summation.The dilated layers reduce channels from 1024 to 256 and operate on single-scale backbone features.
  • Online model update: The complete framework combines offline anchor-free tracking with a plug-in online update model that captures target appearance changes during inference.The online branch inherits early backbone stages and is trained online with a fast conjugate gradient algorithm; IoUNet is not used.
  • Feature combination: Three dilations, (a, b) ∈ {(1, 1), (1, 2), (2, 1)}, capture regions at different scales and improve scale invariance of combined features.
  • Target localization: Target localization uses object-aware anchor-free networks to predict target probabilities from object-aware and regular-region features.The two probabilities are averaged with weight ω before subsequent tracking adjustments.
  • Target localization: Scale-change penalties use previous and current box aspect ratios and sizes, while linear weighting smooths the final predicted scale.The final classification probability is scaled by α, and the final regression scale is β · s′ + (1 −β) · s.

5 Experiments

Ocean is evaluated against state-of-the-art trackers on five benchmarks and through component and training-setting analyses. Results show strong performance across datasets, while ablations assess the contributions of centralized sampling, feature combination, and object-aware features.

  • Evaluation setup: Ocean is evaluated on VOT-2018, VOT-2019, OTB-100, GOT-10k, and LaSOT against 22 representative trackers.The comparison spans anchor-based Siamese, correlation-filter, multi-domain, graph-network, and other tracking methods.
  • State-of-the-art comparison: Ocean ranks second on VOT-2019 overall, while its offline model is best in realtime testing, exceeding DiMPr by 0.6 EAO points and SiamRPN++ by 3.5 points.The passage distinguishes the overall ranking from the realtime comparison.
  • State-of-the-art comparison: 0.592 AO is achieved by offline Ocean on GOT-10k, outperforming SiamRPN++, while the online model improves AO over ATOM by 4.5 points.The online model also surpasses DiMP by 0.9 points in success rate.
  • State-of-the-art comparison: 0.920 precision is achieved by Ocean on OTB-100, while DiMP obtains the best AUC score of 0.686.On LaSOT, offline Ocean reaches 0.527 SUC versus SiamRPN++ at 0.496, and online Ocean reaches the best precision score of 0.566.
  • Component-wise analysis: 0.358 baseline EAO increases by 3.8 points with centralized sampling and by a further 4.2 points after adding the feature combination module.The irregular dilated convolution module provides multi-scale target modeling without much additional computation.
  • Feature visualization: Object-aware features focus on the entire object, whereas regular-region features concentrate on its center, supporting classification and centerness localization respectively.The comparison is visualized on the VOT2018 video “ants1.”

6 Conclusion

Ocean is an object-aware anchor-free tracking framework that directly regresses target positions and uses aligned features for reliable matching. It achieves state-of-the-art performance on five benchmark datasets and suggests parameter updates and other online video tasks as future directions.

  • Ocean directly regresses target positions instead of predicting offsets for predefined anchors.
  • Its alignment module learns an object-aware feature that provides a global target description for reliable object matching.
  • Experiments show state-of-the-art performance on five benchmark datasets.
  • Future work includes updating the object-aware classification network without an additional online branch and applying Ocean to video detection and segmentation.

——————— Supplementary Material ———————–

The supplementary material reports additional ablations and qualitative comparisons. These analyses examine network depth, rectification under drift, and tracker behavior across challenging sequences.

  • Supplementary experiments include additional ablations and qualitative comparisons with state-of-the-art trackers.
  • On OTB-100, increasing convolution layers in the anchor-free regression and classification networks makes AUC become saturated.
  • Rectification capacity is evaluated using mean IoU after sampling weak predictions at controlled shifts from VOT-2018 frames.
  • On six challenging sequences, Ocean better handles fast motion, large deformation, and rotation than the compared trackers in the reported qualitative analysis.
Loading 2006.10721v2…