Transformer Tracking

• Visual object tracking predicts a target’s position and shape in each video frame for applications including robot vision, surveillance, and unmanned driving.Source: §1 · p. 1
• TransT produces more robust and accurate tracking results under occlusion, similar-object interference, and motion blur than two state-of-the-art trackers.Source: Figure 1 · p. 1
• High-accuracy, real-time tracking remains challenging because videos contain occlusion, deformation, and similar-object interference.Source: §1 · p. 1
• Correlation performs local linear matching, losing semantic information and limiting nonlinear interactions between template and region-of-interest features.Source: §1 · p. 1
• TransT introduces attention-based fusion with self-attention ego-context augmentation and cross-attention cross-feature augmentation.Source: §1 · p. 2
• The framework combines feature extraction, attention-only Transformer-like fusion, and prediction heads for classification and regression.Source: §1 · p. 2

• Siamese-based trackers such as SiamFC, SiamRPN, and ATOM commonly use correlation to integrate template information into regions of interest.Source: §2 · p. 2
• Correlation-based networks may underuse global context and lose semantic information, contributing to local optima and imprecise target-boundary prediction.Source: §2 · p. 2
• Transformer attention captures global information by determining which parts of an input sequence are important.Source: §2 · p. 2
• Earlier tracking attention methods improved selection or enhancement but still relied heavily on correlation to fuse template and search-region features.Source: §2 · pp. 2–3
• TransT directly fuses template and search-region features with a new attention-based network without any correlation operation.Source: §2 · p. 3

Overall Architecture

• TransT extracts template and search-region features separately, enhances and fuses them, then predicts foreground/background labels and normalized bounding-box coordinates.The architecture contains a backbone, feature fusion network, and prediction head.Source: §3 · p. 3, §3.1 · p. 4

Feature Fusion Network

• Two ECAs and two CFAs form a fusion layer, which repeats N times before an additional CFA produces the fused feature map.The implementation uses N = 4 fusion-layer repetitions.Source: §3.1 · p. 4

Ego-Context Augment

• ECA uses residual multi-head self-attention with spatial positional encodings to integrate information from different positions within each feature map.The attention output is added residually to the input.Source: §3.2 · p. 4, Figure 3 · p. 4

Cross-Feature Augment

• CFA uses residual multi-head cross-attention and an FFN to adaptively fuse features from the two branches.Queries come from the branch containing CFA, while keys and values come from the other branch.Source: §3.2 · p. 4

Attention Visualization

• Attention visualizations show deeper fusion strengthens target location in search self-attention and target-boundary focus in search cross-attention.Template features increasingly store target-boundary information, while search features retain spatial information.Source: §3.2 · p. 5

Evaluation on TrackingNet, LaSOT and GOT-10k Datasets

• TransT achieves the best performance on LaSOT among compared trackers except SiamR-CNN, while running at 50 fps versus SiamR-CNN’s less than 5 fps.Attribute-based evaluation also reports that TransT performs much better than competing trackers across all LaSOT attributes.Source: §4.2 · p. 7

Ablation Study and Analysis

• TransT without post-processing still achieves state-of-the-art performance, while window-penalty post-processing further improves accuracy across almost all reported metrics.The ablation compares TransT with TransT-np, the version without post-processing.Source: §4.3 · p. 7
• The proposed Transformer-like fusion performs better than the original Transformer structure by a large margin in comparisons with and without post-processing.The comparison keeps the other tracker components, training data, and strategy unchanged.Source: §4.3 · pp. 7–8
• Compared with correlation, attention-based fusion establishes long-distance feature associations and produces semantically rich features rather than only a similarity map.The analysis attributes these properties to global information aggregation between template and search-region features.Source: §4.3 · p. 8, Table 3 · p. 8

Evaluation on Other Datasets

• On NFS, TransT reaches an AUC score of 65.7%, exceeding PrDiMP’s 63.5% by 2.2%; it is comparable on OTB2015 and best on UAV123.The NFS result is calculated from the reported 63.5% baseline and 2.2% gain.Source: §4.4 · p. 8

5. Conclusions

• TransT uses ego-context self-attention and cross-feature cross-attention to fuse template and search-region features and extract richer semantic information.The fusion network is designed to replace correlation for object localization and bounding-box regression.Source: §5 · p. 8
• Experiments across many benchmarks report significantly better performance than state-of-the-art algorithms while the tracker runs at real-time speed.The conclusion characterizes the framework as simple, high-performance, and Transformer-like.Source: §5 · p. 8