Source-linked AI summary
Beyond Human Parts: Dual Part-Aligned Representations for Person Re-Identification
Jianyuan Guo, Yuhui Yuan, Lang Huang, Chao Zhang, Jinge Yao, Kai Han
TL;DR
Person re-identification methods often overlook contextual cues outside predefined human parts or attributes, despite the task's challenging appearance changes. This paper combines human parsing with self-attention in a dual part-aligned representation, achieving new state-of-the-art performance on three benchmarks. Ablation studies report complementary branch contributions, including 1% R-1 and 3% mAP gains when using 5× DPB.
Problem
Existing approaches overlook useful objects and contextual cues outside predefined human parts or attributes, including non-human regions relevant to person re-identification.
Method
The method combines a human parsing branch for accurate predefined human parts with a self-attention latent branch for coarse human and non-human part masks.
Results
The approach achieves new state-of-the-art performances on Market-1501, DukeMTMC-reID, and CUHK03, while 5× DPB yields 1% R-1 and 3% mAP gains over single-branch results.
Takeaways & Limitations
Human-part and latent-part branches provide complementary information, and non-human part context is important for person re-identification.
Abstract
from arXiv · showhide
Person re-identification is a challenging task due to various complex factors. Recent studies have attempted to integrate human parsing results or externally defined attributes to help capture human parts or important object regions. On the other hand, there still exist many useful contextual cues that do not fall into the scope of predefined human parts or attributes. In this paper, we address the missed contextual cues by exploiting both the accurate human parts and the coarse non-human parts. In our implementation, we apply a human parsing model to extract the binary human part masks \emph{and} a self-attention mechanism to capture the soft latent (non-human) part masks. We verify the effectiveness of our approach with new state-of-the-art performances on three challenging benchmarks: Market-1501, DukeMTMC-reID and CUHK03. Our implementation is available at https://github.com/ggjy/P2Net.pytorch.
1. Introduction
Person re-identification must handle substantial appearance changes and part misalignment, while predefined human-part parsing overlooks useful non-human contextual regions. The paper proposes combining accurate human-part representations with coarse latent non-human-part representations.
- Person re-identification is challenged by pose, lighting, occlusion, background clutter, and viewpoint changes that create part misalignment.
- Previous approaches mainly capture predefined human parts through partitioning, attention, attributes, pose estimation, or human parsing.
- Human parsing models divide images into predefined human parts, but may classify crucial objects such as backpacks, reticules, and umbrellas as background.
- The paper learns latent part masks from pixel appearance similarities to capture both human and non-human parts beyond predefined categories.
- The dual part-aligned representation combines accurate human-part information with coarse non-human-part information to augment each pixel representation.
- P2-Net achieves new state-of-the-art performance on Market-1501, DukeMTMC-reID, and CUHK03.
2. Related Work
Related person re-identification methods address part misalignment using handcrafted regions, attention, and semantic attributes, but attention-based approaches generally focus on human parts.
- Handcrafted methods partition images or feature maps into patches or horizontal stripes to extract local region features.
- Attention-based methods learn maps over feature representations to capture human-part information and construct aligned part features.
- Previous attention approaches are described as limited to capturing human parts, leaving non-human contextual information less addressed.
3. Approach
P2-Net augments feature representations with complementary accurate human-part and coarse latent-part information. Its dual part-aligned block applies these branches across the backbone and fuses their outputs with the input features.
- Dual Part-Aligned Representation: The dual part-aligned representation combines accurate human-part information with coarse latent-part information to augment each pixel’s representation.The latent branch captures both coarse human and non-human parts from appearance similarities, while the human branch uses semantic part masks.
- Human Part-Aligned Representation: The human-part branch uses CE2P predictions to form K confidence maps for K−1 predefined human categories plus background.The predicted label map is resized to the feature-map resolution, and each confidence map selects pixels assigned to one category.
- Human Part-Aligned Representation: Each human-part representation aggregates pixel features using normalized confidence maps, then assigns the corresponding part representation to pixels in the aligned feature map.Background pixels aggregate representations of all pixels predicted as background before augmenting their original features.
- P2-Net Architecture: P2-Net inserts a dual part-aligned block after every ResNet-50 stage and adds the human- and latent-aligned outputs to the input feature map.The stages are Res-1 through Res-4, followed by global average pooling and classification.
- Latent Part-Aligned Representation: The latent branch uses self-attention to learn N pixel-specific confidence maps that emphasize pixels belonging to the same latent part according to appearance similarity.This mechanism is intended to capture coarse non-human cues overlooked by predefined human-part approaches.
- Latent Part-Aligned Representation: The latent aligned representation aggregates all pixel features according to their similarities with each target pixel, using a learned transform for the aggregated representation.The similarity transforms are implemented with 1 × 1 convolutions, while the representation transform uses 1×1 convolution, batch normalization, and ReLU.
4. Experiments
Experiments evaluate P²-Net across three person re-identification benchmarks and ablate its human-part and latent-part components. Results show complementary branches, benefits from multiple DPBs, and state-of-the-art performance across Market-1501, DukeMTMC-reID, and CUHK03.
- Datasets and evaluation: Experiments use Market-1501, DukeMTMC-reID, and CUHK03, evaluated with CMC and mAP under the single-query setting.CUHK03 includes both labeled and detected bounding-box variants.
- Implementation: The implementation uses an ImageNet-pretrained ResNet-50 backbone with a 256-D embedding, softmax baseline, and optional triplet loss.Images are resized to 384 × 128 and training uses horizontal flipping and random erasing.
- Ablation study: Larger human-part groupings improve performance, with K = 5 selected as the default human-part setting.The study compares K = 1, K = 2, and K = 5 semantic-part groupings on Market-1501.
- Ablation study: Non-human information accounts for most of the latent-branch gain: Latent w/o HP outperforms Latent w/o NHP and remains close to the original latent branch.The comparison isolates self-attention over non-human versus human regions using parsing-derived binary masks.
- Ablation study: Combining human and latent representations is complementary, while five DPB blocks improve Market-1501 performance by 5.6% R-1 and 11.9% mAP.The five-block configuration is used for the state-of-the-art experiments.
- Benchmark results: P²-Net achieves R-1=95.2% and mAP=85.6% on Market-1501 and surpasses prior state-of-the-art methods on all three benchmarks.On CUHK03, it exceeds MGCAM by 28.2% R-1 and 23.4% mAP, and on CUHK03detected exceeds PCB+RPP by 11.2% R-1 and 11.4% mAP.
5. Conclusion
The paper proposes dual part-aligned representations combining human and latent part branches to address human and non-human part misalignment, achieving state-of-the-art results on three benchmarks.
- The dual part-aligned representation addresses non-human part misalignment in person re-identification.
- Its human part branch uses an off-the-shelf parsing model to capture predefined semantic human parts.
- Its latent part branch uses self-attention to capture detailed part categories beyond injected structural priors.
- The approach achieves new state-of-the-art performances on Market-1501, DukeMTMC-reID and CUHK03.
6. Appendix
The appendix reports implementation details, ablations, complexity comparisons, controlled branch-combination experiments, and qualitative cases illustrating the complementary roles of human and latent part branches.
- Implementation details: Triplet-loss training samples 16 identities with 4 images each and selects hardest positive and negative samples by Euclidean distance.
- Ablation study: DPB performs better when inserted after Res-2 or Res-3 than after Res-1 or Res-4.
- Complexity analysis: The method outperforms ResNet-101 with smaller model size, fewer FLOPs and faster inference on CUHK03.
- Controlled experiments: On DukeMTMC-ReID, combining both branches improves mAP over using only the human part branch, from 66.99 to 67.93 and from 68.64 to 70.84.
- Qualitative analysis: Human-part processing handles misleading car information, while latent masks recover important non-human parts missed by human parsing.
- Controlled experiments: On CUHK03 detected, combining both branches improves mAP from 60.02 to 63.93 and from 63.34 to 64.23.