Distilled Siamese Networks for Visual Tracking

1 INTRODUCTION

• Siamese trackers face high memory costs under real-world budgets, and shrinking then retraining them can cause large tracking-performance degradation.The text notes SiamRPN model size up to 361.8 MB and highlights smartphones as an important constrained setting.Source: §1 · pp. 1–2
• More than 13×–18× compression and nearly 2×–3× speedup are reported for distilled SiamFC and SiamRPN, respectively, with similar tracking accuracy; DSTrpn reaches 265 FPS.On OTB-100, DSTrpn also achieves 3× the speed and 18× memory compression of SiamRPN while retaining comparable accuracy.Source: Figure 1 · p. 1, §1 · p. 2
• DST uses a teacher-student distillation strategy and student-student knowledge sharing to train small, fast, accurate Siamese trackers.The framework models one teacher guiding multiple students that also help one another learn.Source: §1 · p. 2
• Siamese Target Response learning couples the Siamese structure to extract tracking-specific knowledge from teacher networks.The introduction describes it as a transfer-learning approach within the teacher-student module.Source: §1 · p. 2
• Conditional sharing loss encourages reliable knowledge exchange between students and is intended to help with deformation and background clutter.The framework is evaluated on SiamFC, SiamRPN, and SiamRPN++; the paper also reports improved performance on SiamRPN++ backbones.Source: §1 · p. 2

2.1 Visual Tracking and Siamese Trackers

• Siamese methods are described as balancing tracking speed and performance, with later variants improving accuracy or extending the architecture.Examples include SiamRPN++ with a deeper backbone and new correlation method.Source: §2.1 · pp. 2–3
• Siamese trackers match a search region against an initial target exemplar, with SiamFC emphasizing simple, fast online tracking and SiamRPN adding direct location and scale regression.The cited discussion reports nearly 86 FPS for SiamFC and 160 FPS for SiamRPN.Source: §2.1 · pp. 2–3

2.2 Model Compression and Knowledge Distillation

• Pruning, quantization, and knowledge distillation are presented as model-compression approaches, while pruning can require repeated iterations and threshold choices and quantization can perform poorly on large CNNs.Source: §2.2 · p. 3
• Prior distillation work transfers teacher knowledge to students, but DST differs from approaches that learn only a compact backbone by producing more efficient Siamese trackers directly.The discussion also notes applications of distillation to detection and segmentation networks and correlation-filter tracker backbones.Source: §2.2 · p. 3

3 REVISITING SIAMFC AND SIAMRPN

• SiamFC takes a target patch and search region through a two-stream fully convolutional network, then correlates their feature maps to form response map S.The target location is inferred from the response-map peak, and training uses logistic loss with response scores and binary labels.Source: §3 · p. 3
• SiamRPN extends SiamFC with an RPN that outputs foreground-background classification maps and regression vectors for predefined anchors.Template feature maps are used as kernels in cross-correlation to obtain the classification and regression outputs.Source: §3 · pp. 3–4
• SiamRPN training optimizes a multi-task loss combining classification and regression objectives.The classification loss is cross-entropy, while regression uses smooth L1 loss with normalized coordinates.Source: §3 · p. 4

4 DISTILLED SIAMESE TRACKER

• DST selects a reduced student from a teacher tracker via DRL, then trains it simultaneously with an intelligent student through distillation.The framework is organized into architecture selection and teacher-students learning stages.Source: §4 · p. 4

4.1 “Dim” Student Selection

• A policy-gradient agent selects reduced architectures by changing convolution channels, using a reward that balances validation accuracy and compression.The policy searches networks with the teacher’s layers, evaluates top-N proposal overlap on a selected validation subset, and optimizes expected reward over action sequences.Source: §4.1 · p. 4

4.2 Teacher-Students Knowledge Distillation

• TSsKD combines teacher-student knowledge transfer with mutual learning between students to guide the reduced student.The approach is intended to provide more flexible guidance after network selection.Source: §4.2 · p. 5

4.2.1 Teacher-Student Knowledge Transfer

• STR transfers middle-level features and suppresses search-region noise, while teacher-soft and adaptive-hard losses distill teacher outputs and ground-truth supervision.The adaptive-hard regression loss is removed once the student is close to or outperforms the teacher, avoiding over-fitting.Source: §4.2.1 · pp. 5–7

4.2.2 Student-Student Knowledge Sharing

• Students share predictions through conditional weighting that filters unreliable guidance, with KL divergence providing classification supervision between their predicted distributions.Teacher transfer and student sharing run simultaneously, preventing accumulation of representation errors.Source: §4.2.2 · pp. 7–8

5 EXPERIMENTS

• The experiments covered SiamFC, SiamRPN, and SiamRPN++; unless otherwise stated, they used two students and included benchmark evaluations and ablations.For SiamRPN and SiamFC, the authors searched for and trained a suitable dim student.Source: §5 · p. 8

5.3 Benchmark Results

• 265 FPS: on OTB-100, DSTrpn matched or slightly exceeded SiamRPN’s precision and AUC while running nearly 3× faster.DSTrpn also outperformed the compared fast trackers on precision and success plots.Source: §5.3 · p. 9, Figure 6 · p. 9
• On LaSOT and TrackingNet, DSTrpn’s AUC was about 0.02 from its teacher’s, and it outperformed DaSiamRPN on both with a smaller model.Distilled trackers also achieved comparable or slightly better results on FaceTracking.Source: §5.3 · p. 10, Table 3 · p. 10, Table 4 · p. 10

5.4 Qualitative Evaluation

• Biker, Jogging-2, and Subway illustrate DSTrpn outperforming SiamRPN on challenging sequences; it also achieved the best overlap results on all 11 OTB-100 challenges.The cited challenges include deformation and fast motion, occlusion, out-of-plane rotation, and background clutter.Source: Figure 8 · p. 10, §5.4 · p. 11, Figure 9 · p. 11

5.5 Ablation Study

• 15.8% precision and 15.7% AUC gains followed TS loss for SiamRPN; AH+TS+STR performed best among tested transfer variants and closed the teacher–student gap.GT+TS could be suboptimal due to over-fitting; removing the weighting strategy or replacing the distillation loss with bounded loss reduced performance.Source: §5.5 · p. 12, Table 5 · p. 12
• 2.2% and 1.6% AUC gains for dim SiamRPN and SiamFC, respectively, came with TSsKD; it beat TSKD across 11 attributes, by more than 4% on DEF and BC.The intelligent SiamRPN and SiamFC students also improved slightly, by 0.3%.Source: §5.5 · p. 12, Table 6 · p. 12, Figure 10 · p. 12
• 15%–25% grayscale training was suitable; adding students brought no significant gains once one intelligent student was paired with the dim student.On a Xeon 2.20GHz CPU, DSTrpn and DSTfc ran at 20 and 30 FPS, versus 8 and 12 FPS for SiamRPN and SiamFC.Source: Figure 11 · p. 12, §5.5 · p. 13, Table 8 · p. 13

5.6 Extended Experiments on SiamRPN++

• TSsKD significantly improved SiamRPN++ trackers with ResNet34 and ResNet18 backbones across all evaluated benchmarks.The backbone of SiamRPN++ is deep and difficult to train reliably without ImageNet pretraining, so the authors used smaller pretrained backbones as students.Source: §5.6 · p. 13, Table 9 · p. 13

• For SOTA SiamRPN, DST achieved high compression and speed while obtaining performance similar to its teacher.Source: §6 · p. 13
• DST transfers knowledge from a teacher to students using tracking-specific distillation and supports mutual learning among students.Source: §6 · p. 13
• Theoretical analysis and extensive empirical evaluations on two Siamese trackers demonstrated DST's generality and effectiveness.Source: §6 · p. 13
• The authors suggest the distillation method could improve state-of-the-art deep trackers for practical tracking tasks.Source: §6 · p. 13