Source-linked AI summary
Beyond Part Models: Person Retrieval with Refined Part Pooling (and a Strong Convolutional Baseline)
Yifan Sun, Liang Zheng, Yi Yang, Qi Tian, Shengjin Wang
TL;DR
Person retrieval benefits from discriminative part features, but accurately locating parts is difficult, especially without relying on external cues. The paper proposes PCB and refined part pooling to learn consistent part-informed features, achieving 81.6% mAP and 93.8% rank-1 accuracy on Market-1501.
Problem
Discriminative part features require accurately located parts, yet existing strategies rely on external pose cues or omit semantic part information.
Method
PCB uniformly partitions convolutional activations into part-level features, while refined part pooling relocates outliers to closer parts to reinforce within-part consistency.
Results
81.6% mAP and 93.8% rank-1 accuracy on Market-1501 after refined part pooling, improving PCB's 77.4% mAP and 92.3% rank-1 accuracy.
Takeaways & Limitations
PCB is a strong convolutional baseline, and refined part pooling improves it without requiring part labeling information.
Takeaways & Limitations
The approach assumes that feature vectors within a part should be similar and dissimilar from vectors in other parts.
Abstract
from arXiv · showhide
Employing part-level features for pedestrian image description offers fine-grained information and has been verified as beneficial for person retrieval in very recent literature. A prerequisite of part discovery is that each part should be well located. Instead of using external cues, e.g., pose estimation, to directly locate parts, this paper lays emphasis on the content consistency within each part. Specifically, we target at learning discriminative part-informed features for person retrieval and make two contributions. (i) A network named Part-based Convolutional Baseline (PCB). Given an image input, it outputs a convolutional descriptor consisting of several part-level features. With a uniform partition strategy, PCB achieves competitive results with the state-of-the-art methods, proving itself as a strong convolutional baseline for person retrieval. (ii) A refined part pooling (RPP) method. Uniform partition inevitably incurs outliers in each part, which are in fact more similar to other parts. RPP re-assigns these outliers to the parts they are closest to, resulting in refined parts with enhanced within-part consistency. Experiment confirms that RPP allows PCB to gain another round of performance boost. For instance, on the Market-1501 dataset, we achieve (77.4+4.2)% mAP and (92.3+1.5)% rank-1 accuracy, surpassing the state of the art by a large margin.
1. Introduction
The paper targets discriminative, well-aligned part features for person retrieval by emphasizing within-part content consistency rather than relying on external cues. It introduces PCB for uniform part-based convolutional descriptors and RPP for refining inconsistent parts, yielding further gains.
- Person retrieval seeks images of a specified pedestrian from a large database, and deeply learned part features provide discriminative representations.
- Part discovery requires accurate localization, but existing strategies either rely on pose-related external cues or abandon semantic part cues.
- The paper motivates refining coarsely partitioned parts to reinforce within-part consistency and improve spatial alignment.
- 92.3% (+7.0%) rank-1 accuracy and 77.4% (+8.9%) mAP are achieved by PCB on Market-1501, compared with 85.3% rank-1 accuracy and 68.5% mAP for the fully-connected descriptor.
- 93.8% (+1.5%) rank-1 accuracy and 81.6% (+4.2%) mAP are achieved after refined part pooling further improves PCB on Market-1501.
- Combining PCB and RPP achieves a new state of the art, while refined parts are experimentally shown to be superior to attentive parts.
2. Related Work
Related work spans hand-crafted and deeply learned part features, including pose-based parsing and attention-based soft partitioning. RPP is distinguished from PAR by refining pre-partitioned parts rather than directly learning aligned parts.
- Hand-crafted approaches use horizontal stripes, triangles, pictorial structures, or semantic regions to extract local pedestrian features.
- Deep learning improves part features through stronger discriminative representations and tools such as human pose estimation and landmark detection.
- PAR and RPP both use part classifiers for soft partitioning without part labels, but differ in motivation, training, mechanism, and performance.
- PAR learns aligned parts directly with unsupervised attention, whereas RPP refines pre-partitioned parts through a semi-supervised process.
3. PCB: A Strong Convolutional Baseline
PCB modifies a convolutional backbone to produce separately supervised horizontal part features and concatenates them into a convolutional descriptor. Experiments compare its structure with alternative supervision and classifier-sharing designs.
- PCB uses a backbone such as ResNet50 and removes the original global average pooling layer and subsequent components.
- PCB averages backbone tensor columns within p horizontal stripes, reduces each part vector to 256 dimensions, and classifies each part separately.
- During training, PCB minimizes the sum of Cross-Entropy losses over p identity predictions; during testing, concatenated g or h vectors form the descriptor.
- Concatenating G achieves slightly higher accuracy than H but requires greater computation cost.
- PCB performance depends on input size, tensor spatial size, and the number of pooled column vectors.
- The optimized configuration resizes images to 384 × 128, sets tensor T to 24 × 8, and partitions it into 6 horizontal stripes.
- PCB outperforms variants using a single loss or shared classifier parameters, indicating that separate part supervision and classifier weights improve discriminative part features.
4. Refined Part Pooling
Refined part pooling addresses inconsistencies caused by uniform partition by adaptively relocating outlier column vectors according to their similarity to learned parts. It combines a part classifier with probability-weighted sampling and an induced training procedure to refine PCB's hard partition.
- Within-Part Inconsistency: Uniform partition can place outlier column vectors in parts whose contents are less similar than those of another part.The method identifies these outliers by comparing each vector with average-pooled stripe representations using cosine distance.
- Relocating Outliers: Refined part pooling relocates outliers by assigning column vectors to parts according to their predicted similarity.This changes the original hard, uniform partition into a soft, adaptive partition that reinforces within-part consistency.
- Relocating Outliers: A linear layer followed by Softmax predicts each column vector's probability of belonging to each predefined part.PCB uses p = 6 parts, and the trainable weight matrix W defines the part classifier.
- Relocating Outliers: Each refined part is sampled from all tensor column vectors using the predicted part probabilities as sampling weights.The complete set of column vectors in tensor T forms the sampling pool, and the resulting parts replace the original average pooling.
- Induced Training: Induced training first learns standard PCB with uniform partition, then trains the part classifier with PCB layers fixed before fine-tuning the whole network.The procedure uses the pretrained PCB model to induce classifier training; the classifier and subsequent fine-tuning converge in 10 additional epochs in total.
- Induced Training: 80.9% mAP is achieved on Market-1501 with induction, compared with 74.6% mAP without it.The authors report that removing the induction procedure significantly lowers performance and compare the approaches on PCB with refined part pooling.
5. Experiments
Experiments show that PCB is a strong convolutional baseline for person retrieval, while refined part pooling further improves performance, especially on mAP. Results also identify practical design choices for input resolution, down-sampling, and part granularity.
- PCB is a strong baseline: PCB raises mAP over IDE from 68.5%, 52.8% and 38.9% to 77.4%, 66.1% and 54.2% on the three datasets, respectively.The corresponding gains are +8.9%, +13.3%, and +15.3%.
- Refined part pooling: RPP improves PCB’s rank-1 accuracy by +1.5%, +1.6%, and +3.1% and its mAP by +4.2%, +3.1%, and +3.5% across the three datasets.The larger mAP gains indicate particular benefit for finding more challenging matches.
- Comparison with state of the art: PCB surpasses prior methods on Market-1501 using uniform partition alone, and refined part pooling further enlarges the performance lead.The comparison includes methods requiring auxiliary part labels.
- Comparison with state of the art: PCB+RPP exceeds prior mAP by +8.6% on DukeMTMC-reID and +20.5% on CUHK03 under the new training/testing protocol.It also outperforms TriNet+Era and SVDNet+Era, which use extra data augmentation.
- Comparison with state of the art: PCB+RPP reports mAP of 81.6%, 69.2%, and 57.5% and rank-1 accuracy of 93.8%, 83.3%, and 63.7% on Market-1501, DukeMTMC-reID, and CUHK03.These results set new state of the art on all three datasets.
- Parameters Analysis: Halving the down-sampling rate lets PCB with 384 × 128 inputs achieve almost the same performance as 576 × 192 inputs with the original rate.The paper recommends this manipulation with consideration for computing efficiency.
6. Conclusion
The paper presents PCB as a strong baseline for learning part-informed features and introduces refined part pooling to improve within-part consistency without part labels.
- PCB uses a simple uniform partition strategy to learn part-informed features and advances person retrieval performance as a strong baseline.
- Refined part pooling reinforces within-part consistency by grouping similar column vectors into the same part.
- Refined part pooling requires no part labeling information and improves PCB considerably.