PaperlayerView exact revision on arXiv

Source-linked AI summary

Graph Attention Tracking

Dongyan Guo, Yanyan Shao, Ying Cui, Zhenhua Wang, Liyan Zhang, Chunhua Shen

arXiv:2011.11204v1cs.CV

TL;DR

Siamese trackers commonly use fixed-region cross-correlation and global matching, which can retain background, miss foreground, and overlook part-level structure under object variation. SiamGAT uses graph attention over a complete bipartite template–search graph plus target-aware template selection. Across GOT-10k, UAV123, OTB-100, and LaSOT, it achieves leading performance without bells and whistles.

  • Problem

    Fixed-region cross-correlation can retain background or miss foreground, while global matching neglects target structure and part-level information under scale, aspect-ratio, shape, and pose variation.

  • Method

    SiamGAT establishes template–search part-to-part correspondence with a complete bipartite graph, applies graph attention for target-information propagation, and computes a target-aware template area.

  • Results

    SiamGAT outperforms many state-of-the-art trackers and achieves leading performance on GOT-10k, UAV123, OTB-100, and LaSOT.

  • Takeaways & Limitations

    Part-level propagation and adaptive template selection provide the paper's proposed route to more generalizable visual tracking across object-scale and aspect-ratio variations.

  • Takeaways & Limitations

    The graph-attention score is designed under the assumption that search locations more similar to local template features are more likely to be foreground and should receive more target information.

Abstract

from arXiv · show

Siamese network based trackers formulate the visual tracking task as a similarity matching problem. Almost all popular Siamese trackers realize the similarity learning via convolutional feature cross-correlation between a target branch and a search branch. However, since the size of target feature region needs to be pre-fixed, these cross-correlation base methods suffer from either reserving much adverse background information or missing a great deal of foreground information. Moreover, the global matching between the target and search region also largely neglects the target structure and part-level information. In this paper, to solve the above issues, we propose a simple target-aware Siamese graph attention network for general object tracking. We propose to establish part-to-part correspondence between the target and the search region with a complete bipartite graph, and apply the graph attention mechanism to propagate target information from the template feature to the search feature. Further, instead of using the pre-fixed region cropping for template-feature-area selection, we investigate a target-aware area selection mechanism to fit the size and aspect ratio variations of different objects. Experiments on challenging benchmarks including GOT-10k, UAV123, OTB-100 and LaSOT demonstrate that the proposed SiamGAT outperforms many state-of-the-art trackers and achieves leading performance. Code is available at: https://git.io/SiamGAT

1. Introduction

Siamese trackers rely on cross-correlation to embed template and search information, but fixed-region global matching can mis-handle object scale, aspect ratio, shape, and pose variation. SiamGAT addresses these issues with graph-based part-to-part propagation and target-aware template selection, achieving leading benchmark performance.

  • Limitations of Existing Trackers: Cross-correlation Siamese trackers use a pre-fixed target region, which can retain background or miss foreground when object scale or aspect ratio changes.The target feature is commonly obtained by cropping a central m × m template region.
  • Limitations of Existing Trackers: Global matching treats the target as a whole, making similarity computation less robust under rotation, pose variation, shape change, and occlusion.The paper argues that part features are more invariant to shape and pose variation.
  • Proposed Approach: SiamGAT models template–search relations with a complete bipartite graph and uses graph attention to propagate target information between corresponding parts.The graph attention module replaces global information propagation with part-to-part matching.
  • Proposed Approach: A target-aware template computing mechanism adapts template selection to different object sizes and aspect ratios instead of using pre-fixed region cropping.This mechanism is introduced alongside the graph attention module in the SiamGAT framework.
  • Experimental Outcome: Experiments on GOT-10k, UAV123, OTB-100, and LaSOT show that SiamGAT outperforms many state-of-the-art trackers and achieves leading performance.The evaluation uses the simple framework without feature fusion, data enhancement, or other additional performance strategies.

2. Related Work

Related Siamese trackers typically combine feature extraction, similarity matching, and tracking-head decoding, while most prior work focuses on representation or head design. SiamGAT belongs to a line of cross-correlation-based methods but changes how template and search information are embedded.

  • Siamese Tracker Structure: Siamese trackers generally comprise feature extraction, similarity matching for information embedding, and a tracking head for decoding similarity maps.The related-work discussion identifies these as the main structural components.
  • Research Focus: Many prior methods improve Siamese feature representation or tracking-head bounding-box regression, while comparatively little work addresses information embedding.The paper identifies information embedding as an underexplored component.
  • Cross-Correlation Methods: SiamFC introduced Siamese feature extraction with cross-correlation, producing a single-channel similarity map for target-location prediction.Later methods continued using cross-correlation while modifying feature representation or model updating.
  • Region Proposal Trackers: SiamRPN-family trackers add classification and regression branches for anchor proposals, but their performance is sensitive to anchor-related hyperparameters.SiamRPN++ and SiamDW also investigate deeper networks, while C-RPN cascades proposal networks to address data imbalance.
  • Cross-Correlation Variants: SiamRPN++ replaces up-channel correlation with depth-wise cross-correlation to resolve parameter-distribution imbalance between the two branches.This modification targets the training procedure while retaining cross-correlation for information embedding.
  • Alternative Matching: Pixel-to-global matching suppresses background interference but, like cross-correlation, still uses a fixed-scale cropped template feature.The related work presents this as a limitation shared with traditional cross-correlation.

3. Method

SiamGAT combines Siamese feature extraction, graph-attention information propagation, target-aware template selection, and classification-regression localization. Its complete bipartite graph models part-level correspondence while adapting the template feature area to each target’s size and aspect ratio.

  • SiamGAT uses Siamese feature extraction, a graph attention module, and a classification-regression tracking head for target localization.The graph attention module bridges template and search features, while the tracking head predicts classification and bounding-box outputs.
  • Graph Attention Information Embedding: A complete bipartite graph connects template and search feature-map nodes, treating each 1 × 1 × c grid as a target or search part.The graph contains all nodes from the template and search feature maps, enabling part-level correspondence between branches.
  • Graph Attention Information Embedding: Linear transformations followed by inner products produce correlation scores whose softmax-normalized attentions weight target information passed to each search node.The score is intended to reflect similarity between local template and search features, with more similar locations receiving more target information.
  • Graph Attention Information Embedding: Each search-node representation is reconstructed by aggregating information from all neighboring target nodes, producing a response map for subsequent tracking.The aggregated feature is fused with the search-node feature through vector concatenation before downstream processing.
  • Target-Aware Graph Attention Tracking: Target-aware template selection projects the labeled template bounding box onto the feature map, retaining an adaptive region whose dimensions follow the target box.The resulting tensor has dimensions (w, h, c), where w and h correspond to the template bounding-box width and height.
  • Target-Aware Graph Attention Tracking: The selected template region forms the template subgraph, while the search feature map forms the search subgraph; 1×1 convolutions and matrix multiplication improve implementation efficiency.The approach also accommodates non-rectangular regions of interest for tasks such as video instance segmentation.
  • Target-Aware Graph Attention Tracking: The tracking head uses shared GAM response maps in classification and regression branches to predict location category information and bounding boxes.The classification branch predicts category information at each location, and the regression branch computes the target bounding box there.

4. Experiment

Experiments evaluate SiamGAT across multiple tracking benchmarks and ablations of backbone, target-aware template selection, and graph attention matching. SiamGAT achieves leading or competitive performance, including gains over SiamCAR and strong results on unseen classes and long-term tracking.

  • Backbone architecture: Replacing AlexNet with GoogLeNet improves success from 59.2% to 64.6% and precision from 77.9% to 84.3%, while speed decreases from 165 FPS to 70 FPS.The GoogLeNet configuration still meets the reported real-time requirement.
  • Target-aware vs. pre-fixed template area selection: Target-aware template selection adds 2.0% success and 2.1% precision over fixed-region cropping by reducing background information and enhancing foreground representation.The mechanism adapts the selected feature area to the object's aspect ratio.
  • Comparison with DW-Xcorr: With target-aware selection, GAM improves success by 3.1% and precision by 2.8% over DW-Xcorr, whereas fixed-region GAM gains only 1.1% and 0.7%.The comparison identifies fixed target-feature regions as a bottleneck for accurate target information embedding.
  • Evaluation on UAV123: On UAV123, SiamGAT outperforms all compared trackers, improving over SiamCAR by 3.0% in precision and 2.3% in success.The evaluation uses OPE precision and success plots.
  • Evaluation on GOT-10k: On GOT-10k, SiamGAT performs best on all metrics, improving over SiamCAR by 4.8% in AO, 6.6% in SR0.5, and 5.1% in SR0.75.It also exceeds Ocean by 1.6%, 2.2%, and 1.5% on those respective metrics under the benchmark's unseen-class protocol.
  • Evaluation on OTB-100: SiamGAT reaches 71.0% success on OTB-100, surpassing all other trackers, but scores below SiamCAR on fast motion, out-of-view, and low-resolution attributes.It performs better on several shape- and pose-variation attributes, including deformation, rotations, occlusion, illumination variation, and scale variation.
  • Evaluation on LaSOT: On LaSOT, SiamGAT is second only to Ocean-online and surpasses GlobalTrack by 3.6% in normalized precision, 0.2% in precision, and 1.8% in success.It also exceeds the more complex Ocean-offline by 2.3 points in normalized precision, 0.4 in precision, and 1.3 in success.

5. Conclusion

SiamGAT uses graph attention for part-level information propagation and adaptively fits the template area to object scale and aspect ratio. Without additional strategies, it outperforms state-of-the-art trackers across multiple benchmarks.

  • SiamGAT’s graph attention module establishes part-to-part correspondence between template and search regions, enabling search parts to aggregate target information.This replaces traditional cross-correlation-based information embedding with part-level propagation.
  • A recomputed target-aware template area adapts to different object scales and aspect ratios, supporting more generalizable visual tracking.
  • Without bells and whistles, SiamGAT outperforms state-of-the-art trackers by clear margins on GOT-10k, UAV123, OTB-100 and LaSOT.
Loading 2011.11204v1…