Source-linked AI summary
Horizontal Pyramid Matching for Person Re-identification
Yang Fu, Yunchao Wei, Yuqian Zhou, Honghui Shi, Gao Huang, Xinchao Wang, Zhiqiang Yao, Thomas Huang
TL;DR
Person Re-ID remains vulnerable when discriminative body parts are missing, because global features are sensitive to such omissions. HPM learns independently classified partial features across horizontal pyramid scales and combines average and max pooling, achieving state-of-the-art mAP on three benchmarks. Its performance depends on choosing pyramid scales that balance global and local information.
Problem
Person Re-ID approaches remain vulnerable to failure cases where discriminative body parts are missing, while global features are sensitive to missing key parts.
Method
HPM horizontally partitions feature maps at multiple scales, independently classifies each partial representation, and combines average and max pooling for global-local information.
Results
83.1%, 74.5% and 59.7% mAP are achieved on Market-1501, DukeMTMC-ReID and CUHK03, respectively, outperforming state-of-the-art results by more than 1.5%, 5.3% and 2.2%.
Takeaways & Limitations
HPM forms a more robust person representation by exploiting partial information and combining partial-based global average and max pooling.
Takeaways & Limitations
The number of pyramid scales must balance global and local information: too many may underestimate global information, while too few may hinder local-part extraction.
Abstract
from arXiv · showhide
Despite the remarkable recent progress, person re-identification (Re-ID) approaches are still suffering from the failure cases where the discriminative body parts are missing. To mitigate such cases, we propose a simple yet effective Horizontal Pyramid Matching (HPM) approach to fully exploit various partial information of a given person, so that correct person candidates can be still identified even even some key parts are missing. Within the HPM, we make the following contributions to produce a more robust feature representation for the Re-ID task: 1) we learn to classify using partial feature representations at different horizontal pyramid scales, which successfully enhance the discriminative capabilities of various person parts; 2) we exploit average and max pooling strategies to account for person-specific discriminative information in a global-local manner. To validate the effectiveness of the proposed HPM, extensive experiments are conducted on three popular benchmarks, including Market-1501, DukeMTMC-ReID and CUHK03. In particular, we achieve mAP scores of 83.1%, 74.5% and 59.7% on these benchmarks, which are the new state-of-the-arts. Our code is available on Github
Introduction
Person Re-ID must handle visual variation and missing discriminative body parts, which makes global-only representations fragile. HPM addresses this by learning multi-scale horizontal partial features with complementary pooling, improving identification on three benchmarks.
- Global-only person features are sensitive to missing key parts amid variation in pose, clothing, illumination, background, and occlusion.
- HPM horizontally partitions feature maps at multiple pyramid scales and independently classifies the resulting spatial-bin features.Multiple scales introduce slack against misalignment while independently learned parts strengthen scale-specific discrimination.
- HPM combines average pooling for global bin context with max pooling for the most discriminative information in each partition.
- Heatmaps show that HPM identifies more discriminative parts than the non-HPM scheme, leading to better person Re-ID results.
- 83.1%, 74.5% and 59.7% mAP are achieved on Market-1501, DukeMTMC-ReID and CUHK03, outperforming state-of-the-art results by more than 1.5%, 5.3% and 2.2%, respectively.
Related Work
Related Re-ID work learns representations through verification or identification and increasingly exploits local body parts. These part-based methods use pose cues, learned nonsemantic partitions, or predefined patches, while spatial pyramid pooling preserves spatial information across local bins.
- Deep Re-ID methods commonly use verification models that pull same-identity pairs together and push different-identity pairs apart, or identification models that learn person representations.
- Part-based Models: Part-based Re-ID methods generate fine-grained local representations through pose or landmark estimates, nonsemantic partitions, or predefined patches.
- Part-based Models: Pose- and landmark-based methods depend on estimation robustness and may suffer from the gap between pose-estimation and person-retrieval datasets.
- Part-based Models: PCB uses predefined patches for discriminative partition features, but can suffer partition inconsistency from outliers; RPP was introduced to improve within-part consistency.
- Spatial Pyramid Pooling: Spatial Pyramid Pooling produces fixed-length outputs for variable input sizes while retaining spatial information through local spatial bins and robustness to object deformation.
Proposed Method
HPM builds person representations from horizontal spatial bins at multiple pyramid scales, combining global and discriminative local information through average and max pooling. Each part is classified during training, while multiscale part features are concatenated for testing.
- Horizontal Pyramid Pooling: HPM extracts feature maps with a backbone and partitions them into horizontal spatial bins across multiple pyramid scales.The framework uses four pyramid scales in its described implementation, while variants can use different numbers of scales.
- Feature Representation: 2048-dimensional pooled vectors are reduced to 256 dimensions by convolutional layers, producing increasingly detailed partial descriptions at finer scales.Features with the same pyramid-scale index jointly describe the person.
- Classification: Each part feature is fed to a corresponding fully connected classifier and softmax to predict the person identity during training.The framework uses a classification-based objective with a separate prediction for each feature column.
- Testing and Variants: At test time, features from parts at different pyramid scales are concatenated into the final image representation.The number of pyramid scales must balance global information against increasingly fine local partitions.
- Loss Function: The training loss sums cross-entropy losses across the predictions from all spatial bins.This aggregates supervision across the partial representations and pyramid scales.
- Pooling Strategies: Each horizontal bin is pooled with global average and max pooling to capture global-part and most-discriminative-part information.Average pooling preserves global relations, whereas max pooling helps retain highly discriminative regions that averaging with background could weaken.
Experiments
Experiments evaluate HPM on three person Re-ID benchmarks using CMC and mAP, then examine pyramid structure, scale, and pooling choices. HPM achieves strong benchmark results, while ablations support multi-scale partitioning and mixed pooling.
- Results on Market-1501: 83.1% mAP and 94.2% Rank 1 accuracy are achieved on Market-1501, surpassing existing works by more than 1.5% and 0.4%, respectively.The comparison uses HPM without post-processing.
- Results on DukeMTMC-ReID: 74.8% mAP and 86.6% Rank 1 accuracy are achieved on DukeMTMC-ReID without post-processing, exceeding other state-of-the-art methods by 5.3% and 3.3%.The dataset includes eight cameras and substantial variation in person bounding-box size across views.
- Results on CUHK03: 59.7% mAP is achieved on CUHK03 with detected bounding boxes, producing the best mAP under this setting.The Rank 1 accuracy is slightly lower than PCB+RPP, while mAP remains more than 2% higher than other methods.
- Effectiveness of Pyramid Structure: 94.2% Rank 1, 97.5% Rank 5, and 82.7% mAP drop to 92.0%, 96.3%, and 76.4% when the pyramid structure is removed.The non-pyramid variant retains only the branch with eight partial bins.
- Number of Pyramid Scales: Four pyramid scales give the best performance, while increasing scales from one to four improves Rank 1 accuracy from 88.1% to 93.2% and mAP from 71.2% to 79.5%.The number of scales controls the granularity of feature partitioning; one scale is equivalent to global pooling.
- Pooling Strategies: Mixing average and max pooling performs better than using either pooling strategy alone in the ablation results.Average pooling includes all locations equally, whereas max pooling preserves the largest response values; background patterns can influence average pooling.
Conclusion
The work concludes that HPM strengthens partial person representations through global-local pooling and supports further integration into other frameworks and tasks.
- HPM combines partial information with global average and max pooling to improve the robustness of person representations.The approach mines discriminative information from each part in a global-local manner.
- Ablation studies and comparisons demonstrate the effectiveness of the proposed HPM approach.
- The HPM components can be embedded into other frameworks for further performance improvement, with future work targeting related tasks such as human activity recognition.