Source-linked AI summary
Foreground-aware Pyramid Reconstruction for Alignment-free Occluded Person Re-identification
Lingxiao He, Yinggang Wang, Wu Liu, Xingyu Liao, He Zhao, Zhenan Sun, Jiashi Feng
TL;DR
Occlusion and scale variation make person re-identification across disjoint cameras difficult, while prior approaches may require unreliable external cues or advance alignment. The paper proposes FPR, combining FCN-based spatial pyramid features with foreground-weighted reconstruction for alignment-free matching. It reports strong Rank-1 accuracy on three occluded datasets and competitive results on three benchmark datasets.
Problem
Person ReID must match images from non-overlapping cameras despite ubiquitous occlusion, but existing methods can depend on inaccurate external cues or strict alignment.
Method
FPR uses FCN and pyramid pooling to form spatial pyramid features, then applies foreground probability maps to weight reconstruction errors for alignment-free similarity computation.
Results
The method achieves strong results on Partial REID, Partial iLIDS, and Occluded REID, while also achieving competitive results on Market1501, DukeMTMC, and CUHK03.
Takeaways & Limitations
FPR provides an alignment-free occluded-person ReID scheme that can be embedded into end-to-end models and does not require external cues during application.
Takeaways & Limitations
The paper identifies unreliable reconstruction when background and occlusion features are pooled without foreground weighting, which can increase average error and cause mismatching.
Abstract
from arXiv · showhide
Re-identifying a person across multiple disjoint camera views is important for intelligent video surveillance, smart retailing and many other applications. However, existing person re-identification (ReID) methods are challenged by the ubiquitous occlusion over persons and suffer from performance degradation. This paper proposes a novel occlusion-robust and alignment-free model for occluded person ReID and extends its application to realistic and crowded scenarios. The proposed model first leverages the full convolution network (FCN) and pyramid pooling to extract spatial pyramid features. Then an alignment-free matching approach, namely Foreground-aware Pyramid Reconstruction (FPR), is developed to accurately compute matching scores between occluded persons, despite their different scales and sizes. FPR uses the error from robust reconstruction over spatial pyramid features to measure similarities between two persons. More importantly, we design an occlusion-sensitive foreground probability generator that focuses more on clean human body parts to refine the similarity computation with less contamination from occlusion. The FPR is easily embedded into any end-to-end person ReID models. The effectiveness of the proposed method is clearly demonstrated by the experimental results (Rank-1 accuracy) on three occluded person datasets: Partial REID (78.30\%), Partial iLIDS (68.08\%) and Occluded REID (81.00\%); and three benchmark person datasets: Market1501 (95.42\%), DukeMTMC (88.64\%) and CUHK03 (76.08\%)
1. Introduction
The paper targets occluded person ReID without requiring advance alignment or external cues. It combines spatial pyramid features, foreground-aware reconstruction, and foreground weighting, with strong results across occluded and benchmark datasets.
- Motivation: Occlusion in non-overlapping-camera person ReID makes real-world identity matching and movement tracking difficult.The motivating applications include intelligent video surveillance and smart retailing.
- Limitations of Prior Work: Existing methods rely on external masks, parsing, or pose cues that can fail under heavy occlusion, while part-based methods require strict alignment.These external cues also add processing time, according to the paper.
- Proposed Approach: The proposed FPR approach performs alignment-free person re-identification under heavy occlusion using a foreground-aware pyramid reconstruction similarity measure.It is designed to avoid requiring person alignment in advance.
- Proposed Approach: FCN and pyramid pooling generate discriminative spatial pyramid features for person inputs with different sizes and scales.The architecture is described as an end-to-end spatial pyramid feature learning architecture.
- Proposed Approach: Foreground probability maps guide reconstruction by assigning larger weights to body parts and smaller weights to occluded regions.This foreground-aware weighting is intended to reduce contamination from occlusion during matching.
2. Related Work
Prior occluded person ReID work commonly uses external cues or part-to-part matching, whereas FPR avoids both advance alignment and external cues. The related-work comparison frames FPR as an alignment-free alternative for occluded settings.
- Approaches with External Cues: External-cue methods use masks, semantic parsing, or pose information to suppress clutter or locate body parts for person matching.Mask-guided models address background appearance variation, while pose-guided models locate parts using landmarks.
- Limitations of Prior Work: External cues are difficult to acquire accurately and stably when substantial portions of a person are occluded.The paper specifically identifies half-body occlusion as a challenging application condition.
- Proposed Architecture: The proposed architecture combines an FCN, pyramid pooling, and a foreground probability generator to produce spatial pyramid features and foreground maps.These components support the subsequent foreground-aware reconstruction matching process.
- Proposed Alternative: FPR is presented as alignment-free and does not rely on external cues while targeting higher accuracy for occluded-person ReID.This contrasts with prior methods that use external cues for person alignment.
3. Proposed Approach
The proposed model combines fully convolutional spatial features, pyramid pooling, and foreground-aware pyramid reconstruction to match differently sized and occluded persons without alignment. A foreground probability generator weights cleaner regions, while reconstruction errors provide the matching distance and train the end-to-end ReID model.
- Network Architecture: The architecture combines an FCN, pyramid pooling layer, and foreground probability generator for alignment-free occluded person ReID.The FCN retains spatial coordinate information; pyramid pooling produces spatial pyramid features across scales.
- Foreground-aware Pyramid Reconstruction: FPR avoids explicit alignment and accommodates person images with different sizes by reconstructing multi-scale spatial features.The method aggregates probe and gallery spatial features from pyramid maps before computing reconstruction coefficients and residuals.
- Foreground-aware Pyramid Reconstruction: FPR represents each probe spatial feature as a regularized linear combination of gallery spatial features and uses reconstruction error as the matching distance.The coefficient matrix is solved by least squares with ℓ2 regularization, and average reconstruction error measures distance between person images.
- Foreground-aware Pyramid Reconstruction: Foreground probability maps assign larger weights to foreground regions and smaller weights to background or occluded regions during reconstruction.The generator produces a foreground probability vector from spatial probability maps, guiding a weighted sum of reconstruction errors.
- Model Training: The model is optimized with triplet loss and foreground probability generator loss, with FPR embedded after the ReID network during training.The batch-hard triplet loss selects hardest positive and negative samples within randomly formed batches, while the generator learns foreground probability maps.
- Model Training: The foreground-background classifier can accurately detect person parts in foreground probability maps of occluded images.These maps are generated by the foreground probability generator's softmax layer.
4. Experiments
Experiments evaluate FPR on occluded and non-occluded person ReID datasets, compare it with prior methods, and analyze key parameters. Results report strong cross-domain performance and improvements over DSR, with best parameter settings at α = 0.02 and τ approximately 0.35.
- Evaluation on Occluded Person Datasets: FPR is evaluated on Partial REID, Partial iLIDS, and Occluded REID, using established occluded-person datasets with varied viewpoints, backgrounds, and occlusion types.Partial REID contains 600 images from 60 people; Partial iLIDS contains 476 images of 119 people; Occluded REID contains 2,000 images of 200 occluded persons.
- Evaluation on Occluded Person Datasets: FPR retrieval can succeed in cases where DSR fails, using a smaller reconstruction error for the correct result.Figure 6 marks the correct retrieval with a red bounding box.
- Evaluation on Occluded Person Datasets: 81.00%, 68.07%, and 78.30% Rank-1 accuracy are achieved on Occluded REID, Partial iLIDS, and Partial REID, respectively, improving over DSR on each dataset.The reported DSR-to-FPR increases are 72.80% to 81.00%, 64.29% to 68.07%, and 73.67% to 78.30%, respectively.
- Evaluation on Non-occluded Person Datasets: The experiments compare FPR with ten state-of-the-art approaches across part-based, mask-guided, pose-guided, and attention-based categories.The non-occluded evaluation uses Market1501 training data and follows a cross-domain setting for testing.
- Evaluation on Non-occluded Person Datasets: 95.42%, 76.08%, and 88.64% Rank-1 accuracy are reported on Market1501, CUHK03, and DukeMTMC, respectively, with FPR improving over DSR on all three datasets.FPR also raises mAP from 85.78% to 86.58% on Market1501, 71.15% to 72.31% on CUHK03, and 77.07% to 78.42% on DukeMTMC.
- Parameter Analysis: FPR performs best when the foreground-generator loss weight is α = 0.02 and the label threshold τ is approximately 0.35.The parameter study varies α from 0.01 to 0.04 and τ from 0 to 1 on the three occluded person datasets.
5. Conclusions
FPR provides alignment-free matching for occluded person ReID by reconstructing probe spatial features from gallery features and using foreground probabilities to address occlusion. Embedded in batch hard triplet loss, it is validated on occluded datasets and remains competitive on benchmark datasets.
- 5. Conclusions: FPR enables alignment-free matching by linearly reconstructing probe spatial features from gallery spatial features.The method uses reconstruction error to match image pairs without requiring person alignment in advance.
- 5. Conclusions: Spatial foreground probabilities are used during reconstruction to address occlusion.The foreground information focuses reconstruction on cleaner spatial regions of the person.
- 5. Conclusions: FPR is embedded into batch hard triplet loss to learn discriminative features by minimizing same-target reconstruction error and maximizing different-target error.
- 5. Conclusions: Experimental results validate FPR on three occluded person datasets, and the method is competitive on benchmark person datasets.