PaperlayerView exact revision on arXiv

Source-linked AI summary

Body Part-Based Representation Learning for Occluded Person Re-Identification

Vladimir Somers, Christophe De Vleeschouwer, Alexandre Alahi

arXiv:2211.03679v1cs.CV

TL;DR

Occluded ReID needs part-based representations, but local appearance may be non-discriminative and datasets lack human-topology annotations. BPBreID combines dual-supervised body-part attention with GiLt, achieving state-of-the-art results across five datasets and reported gains on Occluded-Duke. Its visibility-score training attempts did not improve performance, remaining a future-research direction.

  • Problem

    Occluded ReID is difficult because local body-part appearance may be non-discriminative and ReID datasets lack human-topology annotations.

  • Method

    BPBreID uses dual-supervised body-part attention and a GiLt training scheme for learning body-part representations robust to occlusions and similar local appearance.

  • Results

    0.7% mAP and 5.6% rank-1 are the reported improvements over state-of-the-art methods on Occluded-Duke, while BPBreID achieves state-of-the-art performance on five popular ReID datasets.

  • Takeaways & Limitations

    The proposed attention and training strategy provides an effective body-part-based approach for occluded and part-based ReID.

  • Takeaways & Limitations

    Accounting for visibility scores during training did not improve performance, although the authors identify it as a promising direction for future research.

Abstract

from arXiv · show

Occluded person re-identification (ReID) is a person retrieval task which aims at matching occluded person images with holistic ones. For addressing occluded ReID, part-based methods have been shown beneficial as they offer fine-grained information and are well suited to represent partially visible human bodies. However, training a part-based model is a challenging task for two reasons. Firstly, individual body part appearance is not as discriminative as global appearance (two distinct IDs might have the same local appearance), this means standard ReID training objectives using identity labels are not adapted to local feature learning. Secondly, ReID datasets are not provided with human topographical annotations. In this work, we propose BPBreID, a body part-based ReID model for solving the above issues. We first design two modules for predicting body part attention maps and producing body part-based features of the ReID target. We then propose GiLt, a novel training scheme for learning part-based representations that is robust to occlusions and non-discriminative local appearance. Extensive experiments on popular holistic and occluded datasets show the effectiveness of our proposed method, which outperforms state-of-the-art methods by 0.7% mAP and 5.6% rank-1 accuracy on the challenging Occluded-Duke dataset. Our code is available at https://github.com/VlSomers/bpbreid.

1. Introduction

Occluded ReID requires representations that avoid misleading occluder information and support matching only mutually visible body parts. BPBreID addresses these challenges with dual-supervised attention and GiLt, reporting gains over prior methods on Occluded-Duke.

  • Challenges: Global ReID features can absorb misleading information from occluding objects and cannot perform part-to-part matching between partially visible images.Part-based approaches instead produce multiple local feature vectors, one for each input part.
  • Challenges: Standard identity and triplet losses are poorly suited to local features because different identities may share similar appearance on individual body parts.This non-discriminative local appearance weakens the assumption that different identities have different feature appearances.
  • Challenges: ReID datasets lack human-topology annotations, while externally generated pose or segmentation labels can be inaccurate and do not directly identify ReID-relevant features.Attention maps must localize body parts and select feature vectors representing discriminative body-part appearance.
  • Proposed approach: BPBreID combines a body-part attention module with GiLt to address occlusion, local appearance, and topology-related challenges.The model is introduced as a body part-based ReID model for the retrieval setting.
  • Results: 0.7% mAP and 5.6% rank-1 are the reported improvements over state-of-the-art methods on Occluded-Duke.The authors also describe dual-supervised soft attention and GiLt as contributions for part-based ReID.

2. Related Work

Related work addresses part alignment and local-feature learning, but existing attention and loss choices leave robustness gaps under occlusion and similar local appearance. GiLt is proposed to address these issues within part-based frameworks.

  • Part-based feature alignment: Fixed attention mechanisms use predetermined image partitions, causing poor feature selection and alignment because they are not designed for ReID-relevant body-part pooling.Other approaches train attention mechanisms end-to-end to generate attention maps.
  • Local feature learning: Identity and batch-hard triplet losses are widely applied to local representations, either independently per part or on combined embeddings.The related methods differ in whether they supervise individual parts or concatenated or summed local features.
  • Local feature learning: Existing objectives such as IHTL and combined-feature losses do not cope well with occluded or similar samples, motivating GiLt as a more robust training procedure.The cited comparison also notes that omitting holistic features reduces robustness to inaccurate part predictions and heavy occlusions.

3. Methodology

BPBreID consists of body-part attention and global-local representation learning modules, with an end-to-end training procedure and an inference procedure for query-gallery distances.

  • Architecture: BPBreID comprises a body-part attention module and a global-local representation learning module.The architecture and procedures are described separately for training and inference.
  • Architecture: The model's overall training procedure and query-to-gallery distance computation are specified as distinct stages.This separates representation learning from inference-time retrieval scoring.

3.1. Body Part Attention Module

The body-part attention module predicts spatial maps for body parts from backbone features. It uses coarse human parsing labels together with ReID identity supervision to produce ReID-relevant attention.

  • Attention prediction: The module applies a pixel-wise part classifier to backbone feature maps and predicts attention maps for body parts.Its ReID loss supplies an additional training signal through identity labels.
  • Attention prediction: A 1x1 convolution followed by softmax produces K+1 maps representing the K body parts and background.The probability maps indicate which pixels belong to each part or the background.
  • Human parsing labels: Human parsing labels are generated with PifPaf and assign each spatial location to a body part or background.For K = 8, the defined regions include head, arms, torso, legs, and feet; Figure 1 uses K = 5.
  • Attention loss: The pixel-wise classifier is trained with a label-smoothed cross-entropy body-part attention loss.The loss uses parsing labels, batch size, smoothing rate, and predicted part probabilities.

3.2. Global-local Representation Learning Module

The global-local representation learning module converts predicted body-part attention maps into holistic and body part-based embeddings, with visibility scores supporting inference-time matching.

  • 3.2. Global-local Representation Learning Module: The module outputs three holistic embeddings and K body part-based embeddings from body-part attention maps.Holistic embeddings are global, foreground, and concatenated representations; body part-based embeddings correspond to the K predicted parts.
  • 3.2. Global-local Representation Learning Module: The architecture combines a body part attention module with a global-local representation learning module.The attention module predicts body-part maps, while the global-local module produces embeddings and visibility scores.
  • 3.2. Global-local Representation Learning Module: Global weighted average pooling produces the foreground embedding and K body part-based embeddings from the appearance feature map.The initial appearance map is also globally average pooled to obtain the global embedding.
  • 3.2. Global-local Representation Learning Module: Visibility scores identify whether each body part is visible and are used only during inference.A part is marked visible when at least one pixel in its attention map exceeds the empirically set threshold λv = 0.4.

3.3. Overall Training Procedure

The overall training procedure uses the GiLt strategy: identity supervision is applied to holistic features, while a part-averaged triplet loss trains body part-based features.

  • 3.3. Overall Training Procedure: The objective combines body part attention loss with GiLt loss, weighting the attention term by λpa = 0.35.The attention loss uses human parsing labels, while GiLt uses identity labels.
  • 3.3.1. GiLt Loss: GiLt combines identity classification loss with a custom part-averaged triplet loss across the model’s embeddings.The strategy requires choosing which loss to apply to holistic and body part-based representations.
  • 3.3.1. GiLt Loss: Identity loss is applied only to holistic features because local body-part features may be insufficiently discriminative for identity prediction.The method argues that applying identity loss to such local representations can damage performance.
  • 3.3.1. GiLt Loss: Triplet loss is applied only to body part-based features because inference directly compares their relative distances.Holistic embeddings are not given the triplet constraint because occlusions can make same-identity holistic representations intrinsically different.
  • 3.3.2. Part-Averaged Triplet Loss: The part-averaged triplet loss averages pairwise distances across corresponding body parts and uses hardest positive and negative samples.This jointly optimizes average local distances rather than imposing a separate triplet constraint on each part.
  • 3.3.2. Part-Averaged Triplet Loss: The averaged-distance design lets each training step focus on the most robust and discriminant parts.The authors report that this choice mitigates the impact of less reliable local features.

3.4. Visibility-based Part-to-Part Matching

At inference, BPBreID computes query-to-gallery distance by matching foreground and body-part embeddings while comparing only mutually visible parts.

  • 3.4. Visibility-based Part-to-Part Matching: Inference uses visibility-based part-to-part matching between a query and gallery sample.The distance uses the foreground embedding together with body part-based embeddings.
  • 3.4. Visibility-based Part-to-Part Matching: Visibility scores ensure that only mutually visible body parts contribute to the comparison.If no body part is mutually visible, the pairwise distance is set to infinity.
  • 3.4. Visibility-based Part-to-Part Matching: Global and concatenated embeddings are excluded from inference because they may include information from occluding objects and pedestrians.

4. Experiments

BPBreID is evaluated across holistic and occluded ReID benchmarks, where it generally ranks first or outperforms prior part-based methods. Ablations support learnable attention, visibility-aware matching, and GiLt's selective loss design.

  • State-of-the-art comparison: BPBreID ranks first overall against compared ReID methods in Table 1.The comparison includes methods with similar ResNet-50 training as well as methods using larger images, stronger backbones, or heavier architectures.
  • State-of-the-art comparison: BPBreID outperforms previous part-based methods on Occluded-Duke and P-DukeMTMC.The authors attribute this to end-to-end attention learning with identity and human parsing supervision, which improves ReID-relevant part pooling.
  • State-of-the-art comparison: BPBreID outperforms all part-based methods on Market-1501 and DukeMTMC-ReID except PFD on Market-1501.PFD uses a substantially heavier ViT-based architecture with a parallel HRNet-W48 pose branch.
  • State-of-the-art comparison: On Occluded-ReID, BPBreID remains competitive without an external inference model or occlusion data augmentation.This dataset has no training set, making domain adaptation important because Market-1501 lacks occluded samples.
  • Ablation studies: Removing learnable attention decreases performance, while ignoring visibility scores dramatically reduces performance through noisy non-visible-part embeddings.These ablations support end-to-end attention learning and visibility-based part-to-part matching.
  • Ablation studies: GiLt performs best when identity and triplet losses are used selectively rather than naively on all embeddings.Holistic embeddings receive identity supervision, while part-based embeddings use the part-averaged triplet loss; the losses are complementary.
  • Embedding analysis: Using all body-part embeddings gives the best embedding performance, with foreground information providing a slight additional improvement.Visibility-weighted matching enables feature alignment, reduces background clutter, and compares only mutually visible body parts; foreground features help when part prediction fails or few parts overlap.

5. Conclusions

BPBreID addresses occluded person ReID by learning body-part representations through dual-supervised attention and the GiLt training strategy, achieving state-of-the-art performance on five popular datasets.

  • BPBreID learns body-part representations for occluded person ReID through a body-part attention module and the GiLt training strategy.The attention module uses identity and human parsing labels, while GiLt trains part-based methods.

Supplementary materials

The supplementary materials detail human parsing-label generation, attention-map and ranking comparisons, and experiments varying body-part definitions and attention learning.

  • Human Parsing Labels: PifPaf produces 17 part confidence fields and 19 part affinity fields used to generate human parsing labels for training.The 36 probability maps highlight human keypoints and joints.
  • Human Parsing Labels: Pixels with no channel value above λt = 0.5 are labeled as background during human parsing-label generation.When multiple people are detected, the target is selected using its head position in the bounding box.
  • Human Parsing Labels: PifPaf-generated labels outperform labels from DensePose and SCHP because of domain transfer and low image quality in ReID datasets.PifPaf provided consistent predictions with few false negatives across image resolutions.
  • Body-Part Configuration: The number K of body parts determines the semantic grouping of human parsing labels and the body-part embeddings generated by the model.Each pixel is assigned background or one of K body-region labels.
  • Qualitative and Ranking Comparisons: The supplementary materials include comparisons of ranking performance and attention maps with other methods.These comparisons are reported in Figures 4 and 5.
  • Qualitative and Ranking Comparisons: Ranking performance is compared between BPBreID, PAT, and BoT, while attention maps are compared between BPBreID, ISP, and PAT.The attention-map comparison includes foreground maps and visibility indications for BPBreID and ISP.
Loading 2211.03679v1…