Source-linked AI summary
Occluded Person Re-identification
Jiaxuan Zhuo, Zeyu Chen, Jianhuang Lai, Guangcong Wang
TL;DR
Occluded person re-identification addresses retrieving a full-body identity from an occluded probe, a setting inadequately covered by conventional re-id. AFPB simulates occlusions and jointly learns identity and occlusion-distribution cues; experiments report superiority across the evaluated datasets and benchmarks.
Problem
Occluded person re-identification retrieves full-body person images from occluded probes, addressing a problem for which few public datasets support deep-learning models.
Method
AFPB uses an Occlusion Simulator to create artificial occlusions and multi-task identification and occluded/non-occluded classification losses to learn robust representations.
Results
AFPB significantly outperforms the baseline, improving rank-1 accuracy by 10.9%, 8.7%, 7.0% and 7.1% on four evaluated datasets.
Takeaways & Limitations
The paper establishes occluded person re-id as a distinct task and provides datasets and a framework designed for occluded-person matching.
Abstract
from arXiv · showhide
Person re-identification (re-id) suffers from a serious occlusion problem when applied to crowded public places. In this paper, we propose to retrieve a full-body person image by using a person image with occlusions. This differs significantly from the conventional person re-id problem where it is assumed that person images are detected without any occlusion. We thus call this new problem the occluded person re-identitification. To address this new problem, we propose a novel Attention Framework of Person Body (AFPB) based on deep learning, consisting of 1) an Occlusion Simulator (OS) which automatically generates artificial occlusions for full-body person images, and 2) multi-task losses that force the neural network not only to discriminate a person's identity but also to determine whether a sample is from the occluded data distribution or the full-body data distribution. Experiments on a new occluded person re-id dataset and three existing benchmarks modified to include full-body person images and occluded person images show the superiority of the proposed method.
1. INTRODUCTION
The paper defines occluded person re-identification as retrieving full-body images from occluded probes and addresses the problem with AFPB, combining simulated occlusions and multi-task learning.
- Motivation: Occlusion is unavoidable in crowded public spaces and removes target information while adding interference from diverse occluders.Occluders vary in color, size, shape, and position, making robust feature learning difficult.
- Problem definition: Occluded person re-identification searches full-body person images using a person image with occlusions as the probe.The paper presents this as distinct from conventional person re-identification assumptions.
- Proposed framework: AFPB uses an Occlusion Simulator to add random background patches to full-body images and generate artificial occluded training data.The artificial occlusion set is jointly used with the source full-body set.
- Proposed framework: AFPB jointly applies identification loss and occluded/non-occluded binary classification loss to learn robust representations and distinguish data distributions.The OBC loss determines whether a sample comes from the artificial occlusion set or source images.
- Contributions: The paper introduces the occluded person re-id problem and releases Occluded-REID, P-DukeMTMC-reID, and P-ETHZ datasets for it.P-DukeMTMC-reID and P-ETHZ are modified from DukeMTMC-reID and ETHZ, respectively.
2. RELATED WORK
Prior person re-identification work extracts representations and learns matching metrics through classification, siamese, or triplet networks, while partial person re-id requires manual cropping and costly patch matching.
- Conventional person re-id: Typical person re-identification separates robust feature extraction from metric learning that brings same-identity distances closer than different-identity distances.Feature extraction addresses illumination, viewpoint, and occlusion, while metric learning improves matching.
- Deep learning methods: Deep person re-id frameworks include classification, siamese, and triplet networks.Classification networks directly extract discriminative features; siamese and triplet networks learn similarity from paired or triplet inputs.
- Partial person re-id: Partial person re-id matches a partial probe with a full-body gallery image but requires manual cropping to remove occlusions.The paper characterizes this operation as unrealistic in practice.
- Partial person re-id: The patch-based partial person matching method requires substantial computation and does not use an attention scheme.AFPB instead directly matches occluded and full-body images while automatically focusing on person-body regions.
3. METHODOLOGY
AFPB addresses occluded person re-identification by generating artificial occlusions and jointly training identity and occlusion-distribution objectives. The framework is designed to learn body-focused representations that remain discriminative for both full-body and occluded images.
- 3. METHODOLOGY: AFPB combines an Occlusion Simulator with multi-task losses to learn robust features for occluded person re-identification.The losses jointly support identity recognition and occluded/non-occluded classification.
- 3.1. Occlusion Simulator (OS): The Occlusion Simulator generates artificial occluded images from full-body data by covering random image positions with resized background patches.The resulting artificial set Z is merged with the original full-body set.
- 3.2. Multi-task losses: The CNN is trained with identification loss so descriptors of the same person become closer while descriptors of different people become more distinct.The full-body set X provides the identity-supervision objective before artificial occlusions are added.
- 3.2. Multi-task losses: The occluded/non-occluded binary classification loss determines whether each sample belongs to the occluded or full-body distribution.Its labels use c = 0 for occluded persons and c = 1 otherwise.
- 3.2. Multi-task losses: The combined objective balances identification and occlusion losses with hyperparameter α, generally using α ≥ 0.5 because identification is the primary loss.The network can use entire person structure information for full-body images and focus on key body parts for occluded images.
- 3.2. Multi-task losses: Saliency maps show that AFPB pays more attention to person body parts rather than occlusions or backgrounds.This supports the framework’s stated goal of learning robust representations against real-world occlusions.
4. EXPERIMENTS
Experiments evaluate AFPB across four datasets containing occluded and full-body images, using occluded probes and full-body galleries. AFPB outperforms baselines and state-of-the-art methods, while its components improve occlusion awareness and body-part focus.
- Experimental setup: The evaluation uses four datasets, each organized into occluded person images and full-body person images, with Occluded-REID newly introduced.The four datasets are Occluded-REID, Partial-REID, P-DukeMTMC-reID, and P-ETHZ.
- Experimental setup: Occluded images serve as probes and full-body images as galleries, with identities split randomly into training and test sets.Both single-shot and multi-shot experiments are reported, and results are averaged over 10 repetitions.
- Comparison with baselines: AFPB improves rank-1 accuracy over the baseline by 10.9%, 8.7%, 7.0%, and 7.1% on the four datasets, respectively.The comparisons include ResNet-50, ResNet-50 with the Occlusion Simulator, and ResNet-50 with multi-task losses.
- Occlusion awareness and attention: OBC classification reaches 88.50%, 85.33%, 91.75%, and 73.88% across the four datasets, demonstrating occlusion awareness.Detection precision over manual body-part annotations exceeds the baseline by 6.21% on Occluded-REID and 7.13% on Partial-REID.
- Comparison with the state-of-the-art: AFPB surpasses the second-best method in rank-1 by 2.35%, 2.33%, 0.97%, and 3.72% on Occluded-REID, Partial-REID, P-DukeMTMC-reID, and P-ETHZ, respectively.The comparison covers four hand-crafted-feature or distance-metric methods and three deep-learning methods.
- Parameter analysis: Performance improves as α increases, with the best performance when α is between 0.7 and 0.9, supporting an auxiliary effect of the OBC loss.α controls the tradeoff between identification loss and OBC loss.
5. CONCLUSION
The paper presents AFPB as a first attempt to address occluded person re-identification by learning robust representations from generated occluded images and multi-task losses. Experimental results show the method's effectiveness and superiority.
- The paper makes the first attempt to solve the occluded person re-identification problem.
- AFPB learns robust feature representations by using generated occluded person images and integrating multi-task losses for person-body attention.
- Experimental results show the effectiveness and superiority of AFPB.