Source-linked AI summary
Siamese Box Adaptive Network for Visual Tracking
Zedu Chen, Bineng Zhong, Guorong Li, Shengping Zhang, Rongrong Ji
TL;DR
Visual trackers often estimate changing target scale and aspect ratio through multi-scale search or heuristic anchor boxes. SiamBAN instead uses an anchor-free fully convolutional network for unified classification and bounding-box regression, achieving state-of-the-art performance across six benchmarks at 40 FPS.
Problem
Existing trackers rely on multi-scale search or carefully designed anchor boxes for scale and aspect-ratio estimation, introducing hyper-parameters and computational complexity.
Method
SiamBAN uses a Siamese backbone with box adaptive heads in a unified FCN that directly classifies targets and regresses bounding boxes without predefined candidate boxes.
Results
SiamBAN achieves state-of-the-art performance on six visual tracking benchmarks and runs at 40 FPS.
Takeaways & Limitations
The no-prior box design avoids candidate-box hyper-parameters, making SiamBAN more flexible and general.
Abstract
from arXiv · showhide
Most of the existing trackers usually rely on either a multi-scale searching scheme or pre-defined anchor boxes to accurately estimate the scale and aspect ratio of a target. Unfortunately, they typically call for tedious and heuristic configurations. To address this issue, we propose a simple yet effective visual tracking framework (named Siamese Box Adaptive Network, SiamBAN) by exploiting the expressive power of the fully convolutional network (FCN). SiamBAN views the visual tracking problem as a parallel classification and regression problem, and thus directly classifies objects and regresses their bounding boxes in a unified FCN. The no-prior box design avoids hyper-parameters associated with the candidate boxes, making SiamBAN more flexible and general. Extensive experiments on visual tracking benchmarks including VOT2018, VOT2019, OTB100, NFS, UAV123, and LaSOT demonstrate that SiamBAN achieves state-of-the-art performance and runs at 40 FPS, confirming its effectiveness and efficiency. The code will be available at https://github.com/hqucv/siamban.
1. Introduction
Visual tracking must estimate changing target scale and aspect ratio amid occlusion, motion, clutter, illumination, and appearance variation. SiamBAN addresses this with an anchor-free FCN that jointly classifies targets and regresses bounding boxes without predefined candidate boxes.
- Visual tracking predicts each subsequent target state from the initial frame despite occlusion, scale variation, background clutter, fast motion, illumination variation, and appearance changes.
- Changing target and camera motion make accurate scale and aspect-ratio estimation a central tracking challenge.
- Existing trackers use multi-scale search or heuristically designed anchor boxes, introducing hyper-parameters and computational complexity.
- SiamBAN uses a Siamese backbone and multiple box adaptive heads in an end-to-end FCN, directly classifying targets and regressing bounding boxes.
- The no-prior box design avoids candidate-box hyper-parameters, making SiamBAN more flexible and general.
- 40 FPS: SiamBAN achieves state-of-the-art results across VOT2018, VOT2019, OTB100, NFS, UAV123, and LaSOT.
2. Related Works
Prior work includes efficient Siamese trackers and anchor-free object detectors, but tracking must handle unknown target categories and distinguish the target from other objects. SiamBAN combines template appearance encoding with anchor-free prediction to address these tracking-specific requirements.
- Siamese network based visual trackers: Siamese network trackers gained attention for end-to-end training and high efficiency, with SiamFC reported to run at 86 FPS.
- Anchor-free object detectors: Anchor-free detectors use FCN-based or point-based designs to predict object boxes without the conventional anchor-box formulation.
- Tracking-specific challenges: Visual tracking differs from detection because target categories are unknown beforehand and the tracker must determine whether objects are the same target.
- Tracking-specific challenges: SiamBAN's Siamese template branch encodes appearance information needed to distinguish the tracked target from other objects.
3. SiamBAN Framework
SiamBAN uses a Siamese backbone with box adaptive classification and regression heads to predict target presence and bounding boxes without anchor boxes. Its point-based labels, multi-level feature fusion, and training/inference procedures support efficient tracking.
- Framework architecture: The Siamese backbone computes template and search features, while box adaptive heads classify foreground versus background and predict bounding boxes at corresponding positions.The two branches share convolutional parameters, and the heads receive adjusted template and search features.
- Framework architecture: SiamBAN predicts four regression values per location and outputs five times fewer variables than anchor-based trackers using five anchor boxes.The classification map has two channels and the regression map has four channels.
- Box adaptive prediction: Each feature-map location corresponds to a search-patch receptive-field center, and regression predicts its distances to the bounding-box sides without shifting that center.An exponential output mapping constrains the regression targets to positive real numbers.
- Multi-level prediction: SiamBAN fuses classification and regression maps from multi-level backbone features using separately learned weights.Earlier features provide fine-grained localization information, while later features provide abstract semantic information robust to appearance changes.
- Ground-truth and loss: Locations inside ellipse E2 receive positive labels, locations outside ellipse E1 receive negative labels, and intermediate locations are ignored for training.Positive locations regress distances to the four ground-truth box sides.
- Training and inference: The multi-task objective combines cross-entropy classification loss with IoU regression loss, using λ1 = λ2 = 1 without hyper-parameter search.During inference, the cached template is matched with each search patch; cosine-window and scale penalties smooth predictions before linear size interpolation.
4. Experiments
Experiments compare SiamBAN with state-of-the-art trackers across multiple benchmarks and examine its robustness, feature aggregation, and label assignment. SiamBAN achieves strong tracking performance while retaining a 40 FPS operating speed.
- Comparison with State-of-the-art Trackers: 40 FPS and state-of-the-art results are reported across VOT2018, VOT2019, OTB100, NFS, UAV123, and LaSOT.The experiments compare SiamBAN with state-of-the-art trackers on six tracking benchmarks.
- Comparison with State-of-the-art Trackers: 8.4% higher EAO and 23.9% lower failure rate than SiamRPN++ are reported on VOT2018.SiamBAN has the highest EAO and ranks second in accuracy among the compared trackers.
- Comparison with State-of-the-art Trackers: SiamBAN ranks first for occlusion, size change, and motion change attributes on VOT2018.It ranks second for camera motion and third for illumination among the evaluated attributes.
- Comparison with State-of-the-art Trackers: 14.7% relative gain in EAO and 17.8% lower failure rate than SiamRPN++ are reported on VOT2019.SiamBAN achieves the highest accuracy and EAO among the compared trackers in the real-time experiment.
- Ablation Study: Aggregating three feature levels produces the best tracking performance in the multi-level prediction ablation.The best two-layer aggregation uses conv4 and conv5 features.
- Ablation Study: Ellipse labels outperform circle and rectangle labels under the same iterations and training dataset.The authors attribute this to more accurate positive and negative sample labeling for foreground-background discrimination.
5. Conclusions
SiamBAN uses a fully convolutional framework to turn visual tracking into unified classification and bounding-box regression without multi-scale search or predefined candidate boxes. Across six benchmarks, it achieves state-of-the-art performance at 40 FPS.
- SiamBAN exploits a fully convolutional network to formulate visual tracking as a classification-regression problem.
- The unified network directly classifies objects and regresses their bounding boxes without multi-scale searching or predefined candidate boxes.
- 40 FPS on six visual tracking benchmarks accompanies SiamBAN's state-of-the-art performance.