Source-linked AI summary
Pose-guided Visible Part Matching for Occluded Person ReID
Shang Gao, Jingya Wang, Huchuan Lu, Zimo Liu
TL;DR
Occluded person ReID must handle appearance disruption from diverse obstacles and the resulting part mismatching. PVPM combines pose-guided attention with a visibility predictor trained from graph-matching pseudo-labels, and reports superior performance on three occlusion datasets.
Problem
Occluded person ReID lacks reliable whole-body appearance because diverse obstacles interfere with global supervision and can cause part mismatching.
Method
PVPM jointly learns pose-guided part features and part visibility, using graph matching to generate pseudo-labels for self-supervised visibility prediction.
Results
PVPM achieves superior performance on Partial-REID, Occluded-REID, and P-DukeMTMC-reID.
Takeaways & Limitations
Pose-guided attention and self-mined visibility are combined to support visible-part matching for occluded person ReID.
Takeaways & Limitations
Ground-truth visibility labels for individual body parts are usually unavailable, so the method relies on correspondence-based assumptions for self-supervision.
Abstract
from arXiv · showhide
Occluded person re-identification is a challenging task as the appearance varies substantially with various obstacles, especially in the crowd scenario. To address this issue, we propose a Pose-guided Visible Part Matching (PVPM) method that jointly learns the discriminative features with pose-guided attention and self-mines the part visibility in an end-to-end framework. Specifically, the proposed PVPM includes two key components: 1) pose-guided attention (PGA) method for part feature pooling that exploits more discriminative local features; 2) pose-guided visibility predictor (PVP) that estimates whether a part suffers the occlusion or not. As there are no ground truth training annotations for the occluded part, we turn to utilize the characteristic of part correspondence in positive pairs and self-mining the correspondence scores via graph matching. The generated correspondence scores are then utilized as pseudo-labels for visibility predictor (PVP). Experimental results on three reported occluded benchmarks show that the proposed method achieves competitive performance to state-of-the-art methods. The source codes are available at https://github.com/hh23333/PVPM
1. Introduction
Occluded person ReID must identify pedestrians despite obstacles that disrupt whole-body appearance and can cause part mismatching. PVPM addresses this with pose-guided feature learning and self-mined visibility supervision, achieving superior performance on three occlusion datasets.
- Motivation: Occlusions from pedestrians and static obstacles violate the assumption that a pedestrian’s entire body is available for matching.Obstacles vary in color, size, and position across real-world scenarios.
- Challenges: Diversified occlusions interfere with global supervision, while visible body parts may be less discriminative than occluded parts, causing mismatching.Non-occluded parts can share similar appearance even when occluded parts contain more discriminative information.
- Approach: PVPM jointly learns discriminative features through pose-guided attention and predicts part visibility in an end-to-end framework.The method combines pose-guided part attention with a pose-guided visibility predictor.
- Approach: PVPM trains visibility prediction with pseudo-labels generated by solving feature correspondence through graph matching.This self-supervised process avoids requiring manually annotated occluded parts.
- Results: PVPM achieves superior performance on Partial-REID, Occluded-REID, and P-DukeMTMC-reID.The reported evaluation covers three occlusion datasets.
2. Related Work
Occluded person ReID research addresses the limitations of holistic representations by incorporating local or occlusion-aware cues. Existing approaches use simulated occlusions, body-part attention, or pose and appearance information, while visibility labels motivate self-supervised learning.
- Occluded Person ReID: Most person ReID methods overlook occlusions, although occlusion is especially relevant in crowded scenes such as airports and hospitals.Prior work includes multi-task learning that distinguishes simulated occluded and non-occluded samples.
- Occluded Person ReID: Co-saliency networks and appearance-based attention exploit body-part information, but local features lack explicit partial-occlusion cues and can mismatch.Other methods incorporate pose information and aggregate parts with bilinear pooling.
- Self-Supervised Learning: Precise body-part visibility labels are unavailable, motivating self-supervised learning for visibility prediction.The related work frames visibility estimation as a setting where manually obtained labels are not readily available.
3. Pose-Guide Visible Part Matching
PVPM combines pose-guided local feature extraction, visibility prediction, and graph-matching-based pseudo-label generation for occluded person ReID. Its training jointly optimizes visibility verification, part matching, and identity classification.
- Framework: PVPM uses a pose encoder, pose-guided attention mask generator, visibility predictor, and feature correspondence model in one framework.The correspondence model generates pseudo-labels for training the visibility predictor.
- Part Features with Pose-Guide Attention: Pose-guided attention extracts part features by combining CNN appearance maps with pose-derived attention maps and retaining complementary nonoverlapping activations.Pose estimation supplies keypoint heatmaps and part affinity fields to the pose encoder and attention generator.
- Pose-Guide Visibility Prediction: The visibility predictor estimates each part’s visibility so matching distances can be computed using only parts visible in both probe and gallery images.The predictor is implemented with global average pooling, a 1×1 convolution, batch normalization, and sigmoid activation.
- Pseudo-Label Estimation by Graph Matching: For positive image pairs, graph matching selects one-to-one corresponding part pairs from node and edge similarities, producing pseudo-labels for visibility learning.The optimal binary matching vector is used as supervision for the visibility predictor.
- Loss Function: The method optimizes visibility verification loss Lv, part-matching loss Lm, and identity classification loss Lc.Visibility learning uses self-supervision, while part matching enhances intra-part consistency and complementary features.
4. Experiments
Experiments evaluate PVPM across three occluded benchmarks, transfer and supervised settings, component ablations, cue types, and parameter choices. PVPM achieves strong rank-1 performance, while analyses support pose-guided attention, visibility weighting, graph matching, part matching loss, and moderate part counts.
- Datasets and Settings: Experiments use Occluded-REID, Partial-REID, and P-DukeMTMC-reID, with Market-1501 used for model pre-training.The reported evaluation includes CMC and mAP under a single-query setting.
- Performance under Transfer Setting: The transfer evaluation directly applies a model trained on Market1501 without further optimization.
- Performance under Transfer Setting: 70.4%, 78.3% and 51.5% rank-1 accuracy are achieved on Occluded ReID, Partial-REID and P-DukeMTMC-reID, respectively.These results are reported under the transfer setting and outperform the compared methods by a large margin.
- Performance under Transfer Setting: 78.3% rank-1 accuracy on Partial-REID surpasses the second-best VPM approach by 10.6%, without requiring test-time image preprocessing.
- Performance under Supervised Setting: 85.1% rank-1 accuracy on P-DukeMTMC-reID under supervised training surpasses the PCB baseline by 5.7%.
- Algorithm Analysis: Ablations show that visibility weighting improves rank-1 by 5.9%, 7.7%, and 3.2% across the three datasets, while removing Lm reduces rank-1 by around 1-2%.Graph matching outperforms thresholding, and augmented occluded samples contribute additional performance gains.
5. Conclusion
PVPM unifies pose-guided attention and part visibility prediction for occlusion ReID, mining visibility scores from target-data correspondences via graph matching. Experiments on three occluded datasets demonstrate the model’s superiority.
- PVPM jointly models discriminative pose-guided attention and part visibility within a unified occlusion ReID framework.
- The method self-mines visibility scores from target-data part correspondences using graph matching, avoiding direct use of visibility cues from other data sources.
- Experimental results on Partial-REID, Occluded-REID, and P-DukeMTMC-reID demonstrate superior performance for occluded person ReID.