Source-linked AI summary
Diverse Part Discovery: Occluded Person Re-identification with Part-Aware Transformer
Yulin Li, Jianfeng He, Tianzhu Zhang, Xiang Liu, Yongdong Zhang, Feng Wu
TL;DR
Occluded person Re-ID must identify people despite missing or noisy visual regions. PAT addresses this with a transformer encoder-decoder and identity-label-guided mechanisms for diverse part discovery, and performs favorably across three Re-ID tasks and six datasets.
Problem
Occlusion introduces noisy regions and makes learning discriminative features from visible parts difficult, while part prototypes lack body-part annotations.
Method
PAT combines a pixel-context transformer encoder, a part-prototype transformer decoder, and part diversity and discriminability mechanisms for weakly supervised part discovery.
Results
PAT performs favorably against state-of-the-art methods on occluded, partial, and holistic Re-ID across six standard datasets.
Takeaways & Limitations
The unified model discovers robust human parts for occluded person Re-ID using only identity labels in a weakly supervised manner.
Abstract
from arXiv · showhide
Occluded person re-identification (Re-ID) is a challenging task as persons are frequently occluded by various obstacles or other persons, especially in the crowd scenario. To address these issues, we propose a novel end-to-end Part-Aware Transformer (PAT) for occluded person Re-ID through diverse part discovery via a transformer encoderdecoder architecture, including a pixel context based transformer encoder and a part prototype based transformer decoder. The proposed PAT model enjoys several merits. First, to the best of our knowledge, this is the first work to exploit the transformer encoder-decoder architecture for occluded person Re-ID in a unified deep model. Second, to learn part prototypes well with only identity labels, we design two effective mechanisms including part diversity and part discriminability. Consequently, we can achieve diverse part discovery for occluded person Re-ID in a weakly supervised manner. Extensive experimental results on six challenging benchmarks for three tasks (occluded, partial and holistic Re-ID) demonstrate that our proposed PAT performs favorably against stat-of-the-art methods.
1 University of Science and Technology of China 2 Dongguan University of Technology
The authors are affiliated with the University of Science and Technology of China and Dongguan University of Technology.
- The affiliation information includes author contact email addresses.
1. Introduction
The introduction motivates occluded person Re-ID, identifies limitations of existing part-based approaches, and presents PAT as a weakly supervised transformer solution for diverse human-part discovery.
- Occlusions from obstacles or other people make holistic person Re-ID assumptions difficult to satisfy.The paper frames effective occluded person Re-ID as essential because cameras may fail to capture the whole pedestrian.
- Occluded regions introduce noise, so robust Re-ID requires discriminative features from unoccluded regions.Existing approaches use local features from human body parts as a mainstream strategy for robust feature learning.
- Hand-crafted splitting, extra-semantic, and attention-based methods respectively face alignment, severe-occlusion, and single-region-focus limitations.The figure examples summarize these limitations across representative part-based strategies.
- PAT uses a pixel-context transformer encoder and part-prototype transformer decoder to discover discriminative parts with identity labels.The encoder captures full-image context, while the decoder uses learnable prototypes to generate part-aware masks and pool part features.
- Part diversity and part discriminability mechanisms guide weakly supervised prototype learning without body-part annotations.The introduction identifies diverse background and absent part annotations as central challenges motivating these mechanisms.
- Experiments cover occluded, partial, and holistic Re-ID across six standard datasets, with results reported as favorable against state-of-the-art methods.The paper presents this evaluation scope as a principal contribution.
2. Related Work
Related work spans holistic, partial, and occluded person Re-ID, while the paper distinguishes PAT from approaches relying on external parsing or pose-estimation models.
- Holistic Person Re-Identification: Prior person Re-ID methods include hand-crafted descriptors, metric learning, deep learning, and part-based feature approaches.
- Partial and Occluded Person Re-Identification: Partial and occluded Re-ID methods use techniques such as pose-guided attention and graph convolution for part alignment and relations.The cited related methods include PVPM and graph convolutional layers for high-order human-part relations.
- Partial and Occluded Person Re-Identification: Many existing methods rely on off-the-shelf human parsing or pose estimators, whereas PAT exploits diverse parts using only identity labels.
3. Part-Aware Transformer
PAT combines a pixel-context transformer encoder with a part-prototype transformer decoder to discover discriminative human parts using identity supervision. The encoder aggregates contextual pixel information, while the decoder produces part-aware masks and features guided by learnable prototypes and diversity and discriminability mechanisms.
- Pixel Context based Transformer Encoder: PAT first extracts a CNN feature map, flattens its spatial dimensions, and applies self-attention to produce a pixel context aware feature map.The encoder models relationships among spatial pixels and aggregates semantically related features.
- Pixel Context based Transformer Encoder: Self-attention aggregates related pixels because same-part features tend to be similar while background pixels differ, improving robustness to background clutter.The encoder then applies a feed-forward network, residual connections, and layer normalization to produce the final contextual feature map.
- Pixel Context based Transformer Encoder: The encoder trains a global feature obtained by global average pooling with identity classification and triplet losses.The triplet formulation distinguishes positive pairs from the same identity and negative pairs from different identities.
- Part Prototype based Transformer Decoder: The decoder uses learnable part prototypes as queries and feature-map pixels as keys and values to generate part-aware masks and weighted part features.Computing across all prototypes yields K attention maps and K corresponding part features.
- Part Prototype based Transformer Decoder: Part diversity discourages prototypes from attending to the same foreground region, while part discriminability uses classification and triplet losses to preserve identity-discriminative part features.The diversity loss expands discrepancies among different part features, and the discriminability losses separate same-prototype features across identities.
4. Experiments
PAT is evaluated across six benchmarks covering occluded, partial, and holistic person Re-ID, with ablations and visualizations examining its components and part discovery.
- Experiments span two occluded, two partial, and two holistic Re-ID datasets, including Occluded-Duke, Occluded-REID, Partial-REID, Partial-iLIDS, Market-1501, and DukeMTMC-reID.
- Occluded Re-ID: 64.5% Rank-1 accuracy and 53.6% mAP on Occluded-Duke, while Occluded-REID reaches 81.6% Rank-1 accuracy and 72.1% mAP.These results set new state-of-the-art performance on both occluded datasets.
- Partial Re-ID: 88.0%/92.3% Rank-1/Rank-3 on Partial-REID and 76.5%/88.2% on Partial-iLIDS, outperforming previous partial person Re-ID models in a cross-domain setting.The model is trained on Market-1501 and tested on the two partial datasets.
- Holistic Re-ID: +3.1% mAP on Market-1501 and +2.6% mAP on DukeMTMC-reID against HOReID, supporting the effectiveness of learnable part prototype learning.
- Ablation Studies: +7.1% mAP from the transformer encoder and +12.9% mAP from adding the part prototype decoder, reaching 51.7% mAP in the reported ablations.The encoder is attributed to pixel-context modeling, while the decoder reduces background influence and aligns part features.
- Part Discovery: The diversity loss yields the full model’s best ablation performance and guides prototypes toward diverse discriminative human parts.Visualized masks from different prototypes focus on distinct regions such as the head and upper body.
5. Conclusion
The paper proposes a Part-Aware Transformer that discovers diverse discriminative human parts using learnable prototypes and identity labels. Experiments across three Re-ID tasks and six datasets demonstrate the method’s effectiveness.
- PAT discovers diverse discriminative human parts using a set of learnable part prototypes for occluded person Re-ID.
- The method learns part prototypes with only identity labels through part diversity and part discriminability mechanisms.
- Extensive experiments across three tasks and six standard Re-ID datasets demonstrate the effectiveness of the proposed method.