Source-linked AI summary
Relation-Aware Global Attention for Person Re-identification
Zhizheng Zhang, Cuiling Lan, Wenjun Zeng, Xin Jin, Zhibo Chen
TL;DR
Person re-identification needs discriminative features despite clutter, pose variation, and occlusion, while prior attention methods largely rely on local convolutions and underuse global structure. The paper proposes Relation-Aware Global Attention, which combines each feature with its pairwise relations to all features and learns attention with a shallow model. RGA improves representation power in ablations and achieves state-of-the-art performance across several person re-identification benchmarks.
Problem
Person re-identification requires discriminative features under background clutter, pose diversity, and occlusion, but prior attention methods typically use local convolutions and overlook mining global structural relations.
Method
RGA stacks each feature node’s pairwise relations with all nodes together with the node’s own feature, then learns spatial or channel attention through a shallow convolutional model.
Results
RGA-SC achieves the best reported performance against other approaches, exceeding the second-best methods by 1.3%/1.3% on CUHK03(L)/CUHK03(D), 0.8% on Market1501, and 0.7% on MSMT17 in mAP accuracy.
Takeaways & Limitations
Global relation representations provide structural information that helps RGA infer attention and focus on discriminative features while suppressing irrelevant ones.
Abstract
from arXiv · showhide
For person re-identification (re-id), attention mechanisms have become attractive as they aim at strengthening discriminative features and suppressing irrelevant ones, which matches well the key of re-id, i.e., discriminative feature learning. Previous approaches typically learn attention using local convolutions, ignoring the mining of knowledge from global structure patterns. Intuitively, the affinities among spatial positions/nodes in the feature map provide clustering-like information and are helpful for inferring semantics and thus attention, especially for person images where the feasible human poses are constrained. In this work, we propose an effective Relation-Aware Global Attention (RGA) module which captures the global structural information for better attention learning. Specifically, for each feature position, in order to compactly grasp the structural information of global scope and local appearance information, we propose to stack the relations, i.e., its pairwise correlations/affinities with all the feature positions (e.g., in raster scan order), and the feature itself together to learn the attention with a shallow convolutional model. Extensive ablation studies demonstrate that our RGA can significantly enhance the feature representation power and help achieve the state-of-the-art performance on several popular benchmarks. The source code is available at https://github.com/microsoft/Relation-Aware-Global-Attention-Networks.
1. Introduction
Person re-identification must extract discriminative identity features despite clutter, pose variation, and occlusion. RGA addresses limitations of locally learned attention by mining global relations and combining them with each feature to infer attention.
- Person re-identification matches a specific person across different times, places, or cameras despite clutter, pose diversity, and occlusion.
- Local convolutional attention has limited receptive fields and struggles to exploit rich structural patterns across the full feature map.Large filters or deeper stacks do not ensure effective global-scope exploration and increase computational or model-size demands.
- Non-local methods aggregate globally using pairwise affinities but do not sufficiently mine the structural information encoded by those relations.Their deterministic weighted-sum use of relations has weak mining capability and limited adaptability.
- RGA stacks each feature node’s relations to all nodes with the node’s own feature, then uses a shallow model to infer attention intensity.This representation combines global structural information with local appearance information for attention learning.
2. Related Work
Attention methods for person re-identification seek to emphasize important features and suppress irrelevant ones, but prior approaches largely rely on local operations or external human semantics. RGA instead learns attention from global relation representations together with local features, mining structural information rather than using relations only for deterministic aggregation.
- Attention matches person re-identification because it emphasizes important features and suppresses irrelevant ones.
- Many prior methods learn attention with convolutions having small receptive fields, while others enlarge context through stacked layers or large filters.
- Pose- or mask-based attention uses explicit human structure but usually requires external annotations or an additional estimation model.
- RGA explores global-scope relations for each feature node, combining affinity and location information to help learn semantics and infer attention.
- Unlike non-local blocks that use pairwise relations as deterministic aggregation weights, RGA mines information from stacked relations through a learned model to derive attention.
3. Relation-Aware Global Attention
RGA learns attention from global relations among feature nodes while retaining each node’s own feature, using shared convolutional transformations for spatial and channel attention. It represents pairwise affinities compactly and applies this design to spatial positions and channels.
- Formulation and Main Idea: RGA contrasts local attention, which ignores global structure, with direct global attention, which is computationally inefficient when the feature count is large.The proposed strategy seeks global awareness without concatenating all features for a large fully connected transformation.
- Spatial Relation-Aware Global Attention: Spatial RGA treats the C-dimensional feature at each spatial position as a node and learns an H × W spatial attention map from spatial affinities.The spatial affinity matrix is built from bidirectional relations between feature nodes.
- Formulation and Main Idea: For each feature node, RGA stacks its pairwise relations with all nodes in a fixed order to form a compact global relation vector.For spatial attention, the vector concatenates the corresponding row and column of the affinity matrix, producing a 2N-dimensional representation.
- Formulation and Main Idea: RGA combines each node’s relation vector with the node feature itself, embeds both domains, and concatenates them before attention prediction.The two inputs provide global structural information and local original information, respectively.
- Channel Relation-Aware Global Attention: Channel RGA treats each channel’s H × W feature map as a node and learns a C-dimensional channel attention vector from channel relations.The channel feature nodes form a graph with C nodes, and their relations are represented by an affinity matrix.
- Analysis and Discussion: Unlike CBAM’s 7 × 7 local neighborhood, spatial RGA uses 1 × 1 convolutions on stacked relations to determine attention from all spatial positions.The comparison highlights global coverage through relation vectors rather than a larger local receptive field.
4. Experiments
Experiments show that RGA modules improve person re-identification performance over the baseline and that global relation representations, asymmetric embeddings, and combined spatial-channel attention contribute to these gains. RGA-SC also achieves the best reported results against competing attention methods and state-of-the-art approaches, while visualizations indicate more discriminative person-region focus.
- RGA Models vs. Baseline: 8.4% higher mAP and 7.3% higher Rank-1 on CUHK03 result from sequential RGA-SC over the baseline.RGA-S and RGA-C separately improve CUHK03 mAP by 5.7% and 6.6%, while Market1501 mAP rises by 3.8% and 4.2%, respectively.
- Ablation of Inputs: Global relation representations improve attention learning, while removing either relations or original features reduces performance.Without the original feature, relation-only variants still exceed the baseline; removing relations makes the corresponding variants inferior to full RGA models.
- Combination Strategies: 2.7% and 1.8% higher CUHK03 mAP make sequential spatial-channel RGA-SC better than RGA-S and RGA-C, respectively.The sequential design lets the later module learn from features modulated by the preceding attention module.
- Visualization: RGA-S learns target-position-adaptive relations that assign larger values to semantically similar positions, unlike the non-local scheme.The relation maps show clustering-like patterns, and Grad-CAM masks indicate better coverage of person regions than the baseline.
- State-of-the-Art Comparison: 1.3%/1.3%, 0.8%, and 0.7% mAP gains over the second-best methods occur on CUHK03(L)/CUHK03(D), Market1501, and MSMT17, respectively.RGA-SC achieves the best performance among the compared approaches on all three datasets and improves over the baseline by 8.4%/9.0%, 4.7%, and 6.0%.
5. Conclusion
The RGA module models global structural information to infer attention and learn more discriminative person re-id features. It is applied across spatial and channel dimensions, with ablations supporting its effectiveness and state-of-the-art performance.
- RGA stacks each feature position’s relations with all features and the feature itself to infer attention using shallow convolutional layers.The representation enables shared kernels across positions while incorporating global structural information.
- The module is applied to both the spatial and channel dimensions of CNN features.
- Extensive ablation studies validate the efficiency of the designs, and the method achieves state-of-the-art performance.