PaperlayerView exact revision on arXiv

Source-linked AI summary

Person Re-identification via Attention Pyramid

Guangyi Chen, Tianpei Gu, Jiwen Lu, Jin-An Bao, Jie Zhou

arXiv:2108.05340v1cs.CVcs.AI

TL;DR

Person re-identification is challenged by pose variation, occlusion, and cluttered backgrounds, while salient focus also changes with visual scale. APNet learns local attention maps, merges them, and stacks attention from coarse to fine. The authors report improved results over state-of-the-art methods across four benchmarks with limited computational cost.

  • Problem

    Person ReID remains challenging because pose variations, occlusions, and cluttered backgrounds cause large intra-class variance, while attention must capture salient clues at different scales.

    Source: §I · p. 1

  • Method

    APNet splits feature maps into local parts, learns and merges their attention maps, then stacks attention at different granularities so coarse attention guides finer learning.

    Source: §I · p. 1, A. Attention Pyramid · p. 3

  • Results

    The authors report that APNet outperforms state-of-the-art methods on four benchmarks with limited computational cost; against APNet0, Market-1501 gains were +2.7% mAP and +1.2% Rank1.

    Source: §I · p. 2, 1) Attention Pyramid Method vs. Baseline · p. 8

  • Takeaways & Limitations

    The attention pyramid is a lightweight plug-and-play module implemented for channel-wise and spatial attention and evaluated on four person ReID benchmarks.

    Source: §I · p. 2

Abstract

from arXiv · show

In this paper, we propose an attention pyramid method for person re-identification. Unlike conventional attention-based methods which only learn a global attention map, our attention pyramid exploits the attention regions in a multi-scale manner because human attention varies with different scales. Our attention pyramid imitates the process of human visual perception which tends to notice the foreground person over the cluttered background, and further focus on the specific color of the shirt with close observation. Specifically, we describe our attention pyramid by a "split-attend-merge-stack" principle. We first split the features into multiple local parts and learn the corresponding attentions. Then, we merge local attentions and stack these merged attentions with the residual connection as an attention pyramid. The proposed attention pyramid is a lightweight plug-and-play module that can be applied to off-the-shelf models. We implement our attention pyramid method in two different attention mechanisms including channel-wise attention and spatial attention. We evaluate our method on four largescale person re-identification benchmarks including Market-1501, DukeMTMC, CUHK03, and MSMT17. Experimental results demonstrate the superiority of our method, which outperforms the state-of-the-art methods by a large margin with limited computational cost.

I. INTRODUCTION

Person re-identification remains difficult because pose variation, occlusion, and cluttered backgrounds create large within-person variation. The introduction proposes coarse-to-fine attention across scales, while the related-work section situates this approach among attention and feature-pyramid methods.

I. INTRODUCTION

  • Person ReID must distinguish people across cameras despite pose variation, occlusion, and cluttered backgrounds that cause large intra-class variance.The task matches images or videos of the same person captured by nonoverlapping cameras.Source: §I · p. 1
  • The motivation is that salient focus changes with scale, shifting from the upper person region in a whole image toward finer discriminative regions at closer view.The proposed process follows human visual attention from coarse to fine.Source: §I · p. 1, Figure 1 · p. 1
  • Attention pyramid networks (APNet) are proposed to jointly learn attention at different scales through a split-attend-merge-stack principle.Features are divided into local parts, their attention maps are merged, and maps from different granularities are stacked.Source: §I · pp. 1–2
  • APNet avoids an extra feature-extraction module, has computational cost comparable to attention learning, and is implemented for channel-wise and spatial attention.The paper evaluates it on Market-1501, DukeMTMC-reID, CUHK03, and MSMT17, and reports improved performance, generalization, and robustness.Source: §I · p. 2

II. RELATED WORK

  • The related-work section reviews deep person re-identification, attention mechanisms, and feature pyramids.These topics frame the proposed multi-scale attention approach.Source: §II · p. 2

A. Deep Person Re-identification

Prior person ReID work improves representations through added knowledge or network design, while attention and feature pyramids offer ways to capture salient or multi-scale information. The approach section introduces APNet implementations for channel-wise and spatial attention.

A. Deep Person Re-identification

  • Person ReID methods use prior knowledge or supervision, including body structure, pose, attributes, and spatial-temporal patterns, as well as network and loss-function designs.Part-based models split feature maps to learn local features but can incur high computational cost.Source: A. Deep Person Re-identification · p. 2

B. Attention Model

  • Attention methods discover salient regions and support discriminative representations, but jointly capturing salient clues at different scales remains little studied.SCSN cascades attention models but requires a complex mechanism to avoid information duplication, increasing computational cost.Source: B. Attention Model · p. 3

C. Feature Pyramid

  • Feature-pyramid methods aggregate multi-scale features, often using an extra branch that increases computational cost; APNet instead learns attention sequentially from coarse to fine.The approach section indicates that APNet is implemented for both channel-wise and spatial attention methods.Source: C. Feature Pyramid · p. 3, §III · p. 3

III. APPROACH

  • The approach section introduces APNet and its channel-wise and spatial attention implementations before describing optimization, implementation details, and comparisons with other pyramid networks.Source: §III · p. 3

A. Attention Pyramid

APNet learns attention maps at progressively finer scales: it splits feature maps into parts, learns and merges local attentions, then stacks attention maps so coarse attention guides finer learning.

A. Attention Pyramid

  • At deeper pyramid levels, coarse attention guides learning of more fine-grained attention maps.The feature maps and attention layers proceed sequentially across levels.Source: A. Attention Pyramid · p. 3
  • The split operation produces increasingly fine-grained parts as pyramid level rises; spatial attention splits along height, while channel-wise attention splits along channels.For radix 2, the number of parts grows as n = 2^i at pyramid level i.Source: A. Attention Pyramid · p. 3
  • Local sub-attention maps are merged by reversing the split operation to form a whole attention map at that level.The merged map indicates salient regions for the level.Source: A. Attention Pyramid · pp. 3–4
  • APNet splits feature maps into multiple parts, learns each part’s attention, merges the maps, and stacks different granularities into an attention pyramid.The resulting attention is applied to the original input feature by element-wise product.Source: Figure 2 · p. 4
  • The attention pyramid is also applied at multiple convolution stages of ResNet50, with four pyramids placed at the bottom of residual blocks for progressive refinement.This multi-stage operation is intended to help the deep network discover salient clues gradually.Source: A. Attention Pyramid · p. 4

B. Attention Model

APNet applies attention at multiple granularities using spatial and channel-wise attention mechanisms. The section describes the underlying attention modules and how local channel attentions are formed and merged.

B. Attention Model

  • Spatial attention learns an H × W map from pairwise feature-node relations to emphasize discriminative regions and suppress irrelevant regions such as occlusion.The baseline is relation-aware attention, whose horizontal and vertical relation maps contribute to the spatial attention.Source: B. Attention Model · p. 4
  • Channel-wise attention uses global average pooling and two fully connected layers to weight salient feature channels.The channel weights are applied to the original feature map by channel-wise multiplication.Source: B. Attention Model · pp. 4–5
  • APNet replaces the original feature map with split sub-features, learns local channel-wise attentions, and merges them into the pyramid.The basic modules include spatial attention based on RGA-S [1] and channel-wise attention based on SE [2], [53].Source: B. Attention Model · p. 5, Figure 3 · p. 5

C. Optimization

APNet is trained with classification and triplet supervision, with label smoothing used to address overfitting. The final objective balances the classification and label-smoothed terms with the triplet loss.

C. Optimization

  • Triplet loss optimizes the embedding space to increase inter-class distance and reduce intra-class distance.The authors use hard-example mining, Euclidean distance, and a positive-sample margin of m = 0.3 relative to negative samples.Source: C. Optimization · p. 5
  • The classification loss uses cross-entropy between identity ground truth and predicted probabilities, with label smoothing used to address overfitting.The smoothing uses a uniform distribution μ(k) = 1/K; ε is the smoothing rate.Source: C. Optimization · p. 5
  • The final training objective combines classification and label-smoothed loss with a weighted triplet-loss term.λ is the balance rate between the two loss functions.Source: C. Optimization · p. 5

D. Discussion

The discussion distinguishes APNet from feature-pyramid approaches by using multi-granularity splits and stacked attention rather than extra feature extraction. It contrasts APNet's attention pyramid with FPN, Pyramid ReID, DPD, and feature pyramid attention.

D. Discussion

  • Unlike FPN, which aggregates different-resolution features through a costly top-down branch, APNet places its pyramid structure in the backbone as an attention model.The discussion argues that identity information matters more than resolution for ReID feature representations.Source: D. Discussion · p. 5
  • The figure contrasts FPN's resolution aggregation, Pyramid ReID's local-feature branches, feature pyramid attention's multi-scale convolutions, and APNet's split-attend-merge-stack design.The APNet design is described as coarse-to-fine attention learning without extra feature-extraction cost.Source: Figure 4 · p. 6
  • Unlike Pyramid ReID, which extracts local features through network branches, APNet stacks attention layers to reduce redundant computation and guide fine attention with coarse maps.The comparison also distinguishes DPD: APNet connects scales with stacked attention and uses attention rather than pooling to obtain pyramidal features.Source: D. Discussion · pp. 5–6
  • Unlike feature pyramid attention [60], which fuses multi-scale information to learn one attention, APNet learns multi-scale attentions through multi-granularity splitting.The passage characterizes APNet as requiring no extra computational cost and as integrable with different attention modules.Source: D. Discussion · p. 6

IV. EXPERIMENTS

The experiments evaluate APNet across ReID benchmarks, with ablations and comparisons to state-of-the-art methods. The dataset-settings subsection identifies the benchmarks and the ablation table's scope.

IV. EXPERIMENTS

  • The study evaluates APNet on Market-1501, DukeMTMC-reID, CUHK03, MSMT17, and Occluded-DukeMTMC-reID, and also conducts cross-dataset experiments.The experiments include ablations and comparisons with state-of-the-art methods, as well as evaluations of occlusion robustness and generalization.Source: §IV · p. 6

A. Datasets and Experimental Settings

  • The dataset-settings subsection lists Market-1501, DukeMTMC-reID, CUHK03, MSMT17, and Occluded-DukeMTMC as the five experimental datasets.The dataset details are presented in Table I.Source: A. Datasets and Experimental Settings · pp. 6–7
  • The APNet-C ablation table covers Market-1051, DukeMTMC-reID, and MSMT17.The supplied table caption identifies the method and datasets but does not provide result values.Source: Table II · p. 7

1) Market-1501:

The experiments use four person re-identification benchmarks, standard CMC and mAP evaluation, and a shared ResNet50-based setup with attention-pyramid variants.

1) Market-1501:

  • Market-1501 has 751 training and 750 test identities, with 12,936 training and 19,732 test images; evaluation follows the single-query protocol.Images come from five high-resolution and one low-resolution camera.Source: 1) Market-1501 · p. 7

2) DukeMTMC-reID:

  • DukeMTMC-reID uses 702 identities for training and 702 for testing, with 16,522 and 19,889 images, respectively.Each identity is observed by two cameras among eight high-resolution cameras.Source: 2) DukeMTMC-reID · p. 7

3) CUHK03:

  • CUHK03 experiments use both manually labeled and auto-detected person boxes and the CUHK03-NP split, with 767 identities for training and 700 for testing.The paper describes CUHK03-NP as more realistic and challenging than the 1,367/100 split.Source: 3) CUHK03 · p. 7

4) MSMT17:

  • MSMT17 contains 126,441 images of 4,101 identities, split into 32,621 training images and 93,820 test images.The images were recorded across 15 cameras, including indoor and outdoor cameras, over four days with different weather conditions.Source: 4) MSMT17 · p. 7
  • CMC measures whether query identities appear among the top N results, while mAP reflects overall re-identification accuracy across the precision-recall curve.The experiments use Euclidean distance without re-ranking.Source: 4) MSMT17 · p. 7

B. Implementation Details

  • Experiments use ImageNet-pretrained ResNet50 with APNet added after each residual block, implementing both channel-wise and spatial attention pyramids.Training uses dataset-specific input sizes and 160 epochs; a multi-part trick is applied only to CUHK03.Source: B. Implementation Details · p. 7

C. Ablation Studies

The ablations compare attention-pyramid variants with no-attention and naive-stacking baselines, testing the pyramid structure and its level choices.

C. Ablation Studies

  • The ablation compares spatial and channel-wise APNet at different pyramid levels with APNet0, stacked attention, and original SE-ResNet.APNet0 has no attention, while stacked attention repeats the same attention without splitting.Source: C. Ablation Studies · p. 7

1) Attention Pyramid Method vs. Baseline:

  • APNet-S and APNet-C improve over APNet0: Market-1501 gains +2.7%/+1.2% and DukeMTMC-reID +4.3%/+2.0% mAP/Rank1; MSMT17 gains +12.3%/+8.4% and +7.8%/+5.4%, respectively.The two MSMT17 gain pairs are for channel-wise and spatial APNet, respectively.Source: 1) Attention Pyramid Method vs. Baseline · p. 8

2) Attention Pyramid vs. Stacked Attention:

  • With three levels, APNet-C3 beats stacked attention by +0.9%/+0.8% on Market-1501, +2.6%/+1.8% on DukeMTMC-reID, and +7.9%/+7.5% on MSMT17 mAP/Rank1.The authors report significant improvement for spatial APNet as well and attribute the gains to splitting rather than simply adding attention layers and parameters.Source: 2) Attention Pyramid vs. Stacked… · p. 8

3) Influence of Pyramid Level:

  • APNet-C2 exceeds APNet-C1 by +0.9%/+0.7% on Market-1501, +2.5%/+1.8% on DukeMTMC-reID, and +4.0%/+1.8% on MSMT17 mAP/Rank1.Further pyramid depth brings slower improvement; the authors select a low-level pyramid for subsequent experiments because deeper levels add computation for trivial gains.Source: 3) Influence of Pyramid Level · p. 8

4) Channel-wise Attention vs. Spatial Attention:

APNet is compared with prior methods across four benchmarks and evaluated for attention type, split radix, computational cost, and backbone generalization.

4) Channel-wise Attention vs. Spatial Attention:

  • Channel-wise and spatial attention perform similarly on Market-1501 and DukeMTMC-reID, while channel-wise attention has higher MSMT17 accuracy at all three pyramid levels.The channel-wise Rank-1/mAP advantages are 3.7%/3.7%, 4.6%/2.9%, and 4.1%/2.1% at levels 1, 2, and 3.Source: 4) Channel-wise Attention vs. Spatial… · p. 8

5) Influence of Split Radix:

  • With level-2 attention pyramid networks as the baseline, comparison of split radixes r = {1, 2, 4, 8, 16, 32} found mAP and Rank-1 accuracy maximal at radix 2, then slightly lower at higher radixes.The authors attribute the decline to lower granularity hindering discovery of discriminative semantic clues and recommend maintaining an adequate splitting scale.Source: 5) Influence of Split Radix · pp. 8–9

6) Analysis of computational cost:

D. Comparison to State-of-the-Art Methods

  • On Market-1501, APNet improves over RGA-SC by 1.1% mAP and 0.1% Rank-1, and over Pyramid ReID by 2.3% mAP and 0.5% Rank-1.It also saves 37.4% computational cost versus Pyramid ReID.Source: D. Comparison to State-of-the-Art… · p. 9
  • On DukeMTMC-reID, APNet exceeds ISP by 1.5%/0.5% mAP/Rank1 and matches SCSN's Rank-1 while improving mAP by 2.5% with 14.0% less computation.SCSN has the state-of-the-art Rank-1 performance in this comparison.Source: D. Comparison to State-of-the-Art… · p. 9
  • On CUHK03, APNet reaches 85.3% mAP and 87.4% Rank-1 on labeled data, and 81.5% mAP and 83.0% Rank-1 on detected data.The paper reports better performance than SCSN; its multi-part trick is used only on CUHK03 because its improvement is limited on larger datasets.Source: D. Comparison to State-of-the-Art… · pp. 9–10
  • APNet achieves superior performance over the listed methods on Market-1501, DukeMTMC-reID, CUHK03, and MSMT17.Table IV summarizes the comparisons, including detected and labeled CUHK03 results.Source: D. Comparison to State-of-the-Art… · p. 9, Table IV · p. 10
  • On MSMT17, APNet obtains 63.5% Rank-1 and 83.7% mAP, exceeding ABD-Net by 2.7% and 1.4%, respectively.Source: D. Comparison to State-of-the-Art… · p. 10

E. Robustness Analysis

The robustness evaluation tests APNet on Occluded-DukeMTMC, where query images are fully occluded, and reports higher accuracy than the compared methods.

E. Robustness Analysis

  • Occluded-DukeMTMC has 9% occluded training images, 100% occluded query images, and 10% occluded gallery images.The original DukeMTMC-reID split has 14%, 15%, and 10% occluded images in training, query, and gallery, respectively.Source: E. Robustness Analysis · p. 11
  • On Occluded-DukeMTMC, APNet achieves 54.2% mAP and 62.2% Rank-1, exceeding the current best model by 10.3% and 7.1%, respectively.The query set contains 100% occluded images.Source: E. Robustness Analysis · p. 11
  • The authors report that APNet-C2 improves substantially over APNet-C1, which they interpret as evidence that the pyramid captures multi-scale salient clues that further prevent absorbing occluded features.They describe this dataset as a direct test of models trained on original images and evaluated on occluded images.Source: E. Robustness Analysis · p. 11

F. Generalization Ability Analysis

Cross-dataset evaluation reports improvements over the baseline in both transfer directions, while Grad-CAM visualizations show attention concentrating on salient person features at deeper pyramid levels. The conclusion describes APNet as effective, efficient, and robust compared with other pyramid structures or attention models.

F. Generalization Ability Analysis

  • +5.7%/+6.8% mAP/Rank-1 from Market to Duke and +4.6%/+6.4% mAP/Rank-1 from Duke to Market over the baseline.The tests trained on Market-1501 and evaluated on DukeMTMC-reID, and vice versa; APNet was comparable to SCAL on Duke→Market and significantly improved Market→Duke performance.Source: F. Generalization Ability Analysis · pp. 11–12

G. Qualitative Analysis

  • Grad-CAM visualizations show attention becoming more concentrated on salient person features, including handbags, umbrellas, clothing logos, and shirts, at deeper pyramid levels.The authors report that this concentration alleviates the common misalignment issue in image retrieval.Source: G. Qualitative Analysis · p. 12

V. CONCLUSION

  • APNet stacks local attentions from feature splits of different granularities, using coarse attention to guide fine-grained learning in spatial and channel-wise modules.The conclusion states that the module can be integrated into any attention model and is more effective, efficient, and robust than other pyramid structures or attention models.Source: §V · p. 12

Related papers

Loading 2108.05340v1…