PaperlayerView exact revision on arXiv

Source-linked AI summary

Two-stream Beats One-stream: Asymmetric Siamese Network for Efficient Visual Tracking

Jiawen Zhu, Huayi Tang, Xin Chen, Xinying Wang, Dong Wang, Huchuan Lu

arXiv:2503.00516v1cs.CV

TL;DR

Efficient tracking seeks practical operation on resource-constrained platforms, but one-stream designs repeat template computation and impose intensive template–search interaction. AsymTrack separates the streams, using one-time template computation, modulation, and object-perception enhancement. It reports improved speed–precision trade-offs across platforms, including higher LaSOT AUC than HiT-Tiny alongside faster CPU and AGX speeds.

  • Problem

    Efficient trackers often adopt one-stream designs that repeat template computation and perform intensive template–search relation modeling, while practical tracking requires low latency on resource-constrained platforms.

    Source: Introduction · pp. 1–2

  • Method

    AsymTrack separates template and search branches, computes the template once to generate modulation signals, and combines semantic and local cues through an object-perception enhancement module.

    Source: Introduction · p. 2, Object Perception Enhancement · p. 5

  • Results

    6.0% higher AUC on LaSOT than HiT-Tiny, with faster CPU and Jetson AGX Xavier speeds.

    Source: Introduction · p. 2

  • Takeaways & Limitations

    AsymTrack is presented as an alternative to one-stream architectures for real-time visual tracking on resource-constrained platforms.

    Source: Conclusion · p. 7

Abstract

from arXiv · show

Efficient tracking has garnered attention for its ability to operate on resource-constrained platforms for real-world deployment beyond desktop GPUs. Current efficient trackers mainly follow precision-oriented trackers, adopting a one-stream framework with lightweight modules. However, blindly adhering to the one-stream paradigm may not be optimal, as incorporating template computation in every frame leads to redundancy, and pervasive semantic interaction between template and search region places stress on edge devices. In this work, we propose a novel asymmetric Siamese tracker named \textbf{AsymTrack} for efficient tracking. AsymTrack disentangles template and search streams into separate branches, with template computing only once during initialization to generate modulation signals. Building on this architecture, we devise an efficient template modulation mechanism to unidirectional inject crucial cues into the search features, and design an object perception enhancement module that integrates abstract semantics and local details to overcome the limited representation in lightweight tracker. Extensive experiments demonstrate that AsymTrack offers superior speed-precision trade-offs across different platforms compared to the current state-of-the-arts. For instance, AsymTrack-T achieves 60.8\% AUC on LaSOT and 224/81/84 FPS on GPU/CPU/AGX, surpassing HiT-Tiny by 6.0\% AUC with higher speeds. The code is available at https://github.com/jiawen-zhu/AsymTrack.

Introduction

Visual tracking needs lower-latency operation on resource-constrained platforms, but one-stream efficient trackers repeat template computation and use pervasive template–search relation modeling. AsymTrack uses an asymmetric Siamese design with template modulation and object-perception enhancement, and reports stronger speed–precision trade-offs than competing trackers.

Introduction

  • One-stream efficient trackers repeat template computation at every frame and rely on pervasive template–search relation modeling that stresses edge devices.These costs motivate a design that avoids repeated template computation while retaining relation modeling.Source: Introduction · p. 2
  • AsymTrack separates template and search branches, transforms template features into modulation signals, and injects them into the search branch for relation modeling.The template branch is not repeatedly computed during inference.Source: Introduction · p. 2
  • 6.0% higher AUC on LaSOT than HiT-Tiny, with faster CPU and Jetson AGX Xavier speeds; AsymTrack-B also runs 6.6 times faster on CPU and 4.7 times faster on AGX than TransT at comparable AUC.AsymTrack-B reports 64.7% AUC versus TransT’s 64.9%.Source: Introduction · p. 2
  • The proposed ETM and OPE modules support relation modeling and improve lightweight trackers’ limited object representation, respectively.The paper presents AsymTrack as combining two-stream efficiency with one-stream precision benefits.Source: Introduction · p. 2

Related Works

Precision-oriented tracking has shifted from Siamese feature fusion toward one-stream transformer designs, while efficient tracking seeks to reduce computational demands for practical deployment. The related work reviews lightweight approaches in both paradigms and notes the efficiency challenges that motivate asymmetric Siamese tracking.

Precision-Oriented Tracking

  • One-stream trackers such as OSTrack and MixFormer became the mainstream precision-oriented paradigm, while their performance-focused designs can miss speed requirements on resource-constrained platforms.ARTrack, for example, runs at 9 FPS on CPUs and 8 FPS on AGX.Source: Precision-Oriented Tracking · p. 2

Efficiency-Oriented Tracking

  • AsymTrack’s overview shows an asymmetric Siamese pipeline in which the template branch runs once at initialization and feeds features and a prototype to the search branch for online inference.This design is presented as an alternative that combines strengths of the two tracking paradigms.Source: Figure 2 · p. 3, Efficiency-Oriented Tracking · p. 3
  • Efficient trackers use lightweight designs in Siamese and one-stream frameworks, but one-stream approaches can repeat template computation and incur costly transformer attention.MixFormerV2 lightens a one-stream transformer for real-time CPU speed, while HiT uses lightweight hierarchical transformers and shallow features for edge speed.Source: Efficiency-Oriented Tracking · pp. 2–3

Methodology

AsymTrack separates template and search processing, uses template-derived modulation to communicate cues to the search stream, and enhances lightweight features with semantic and local descriptors. Its evaluations compare accuracy and speed across multiple benchmarks and platforms.

Preliminaries and Notation

  • Siamese trackers interact separate template and search features, whereas one-stream trackers concatenate their tokens for transformer-based interaction.Source: Preliminaries and Notation · p. 3

Asymmetric Siamese Architecture

  • AsymTrack uses separate parameter-sharing encoders, with the template encoder a subset of the search encoder, and sends initialization-time template prototypes into the search branch.Source: Asymmetric Siamese Architecture · pp. 3–4

Efficient Template Modulation

  • ETM extracts compact template and search prototypes, uses 1D dynamic convolution for prototype relation modeling, then applies prototype attention to modulated search cues.Source: Efficient Template Modulation · p. 4

Object Perception Enhancement

  • OPE combines 3×3 semantic and 1×1 gradient-derived local descriptors; re-parameterization folds them into a single 3×3 convolution without performance degradation.Source: Object Perception Enhancement · p. 5

Optimization and Tracker Inference

Comparison with State-of-the-arts

Exploration Studies

  • Ablations report gains from template modulation and OPE, minimal OPE latency after re-parameterization, effective prototype attention, little benefit from early-stage ETM, and best results with OPE at every stage.Source: Exploration Studies · p. 7, Table 5 · p. 7

Conclusion

AsymTrack presents an asymmetric Siamese family of efficient tracking models as an alternative to one-stream architectures. The paper positions the family for real-time tracking on resource-constrained platforms.

  • AsymTrack is presented as a new family of efficient tracking models.Source: Conclusion · p. 7
  • Its asymmetric Siamese framework combines the efficiency of two-stream trackers with the performance benefits of one-stream designs.Source: Conclusion · p. 7
  • The family offers a viable alternative to one-stream architectures for real-time tracking on resource-constrained platforms.Source: Conclusion · p. 7

Related papers

Loading 2503.00516v1…