Source-linked AI summary
Deeply-Learned Part-Aligned Representations for Person Re-Identification
Liming Zhao, Xi Li, Jingdong Wang, Yueting Zhuang
TL;DR
Person re-identification must associate people across cameras despite body-part misalignment caused by pose changes and varying positions in bounding boxes. The paper learns an attention-inspired, part-aligned representation from person similarities, and reports competitive or superior results across standard datasets.
Problem
Person re-identification is challenging because pose changes and varying human spatial distributions undermine global or fixed spatial-partition representations.
Method
A deep network detects discriminative body parts, computes part representations, aggregates corresponding-part similarities, and learns these components jointly without body-part labels.
Results
The approach achieves competitive or superior performance on Market-1501, CUHK03, CUHK01, and VIPeR.
Takeaways & Limitations
Learning parts directly for person re-identification produces representations intended to be more robust to pose changes and different spatial distributions than grid- or stripe-based partitioning.
Takeaways & Limitations
Performance comparisons indicate that training deep networks from small person re-identification datasets remains an open and challenging problem.
Abstract
from arXiv · showhide
In this paper, we address the problem of person re-identification, which refers to associating the persons captured from different cameras. We propose a simple yet effective human part-aligned representation for handling the body part misalignment problem. Our approach decomposes the human body into regions (parts) which are discriminative for person matching, accordingly computes the representations over the regions, and aggregates the similarities computed between the corresponding regions of a pair of probe and gallery images as the overall matching score. Our formulation, inspired by attention models, is a deep neural network modeling the three steps together, which is learnt through minimizing the triplet loss function without requiring body part labeling information. Unlike most existing deep learning algorithms that learn a global or spatial partition-based local representation, our approach performs human body partition, and thus is more robust to pose changes and various human spatial distributions in the person bounding box. Our approach shows state-of-the-art results over standard datasets, Market-$1501$, CUHK$03$, CUHK$01$ and VIPeR.
1. Introduction
Person re-identification becomes difficult when cameras are disjoint and temporal transitions vary, motivating appearance-based representations that remain reliable under body misalignment. The paper proposes learning discriminative, body-part-aligned representations rather than relying on fixed spatial partitions.
- Disjoint camera views and variable transition times make temporal information insufficient for associating the same person across cameras.
- Body-part decomposition avoids the mismatches illustrated by fixed spatial partitioning, where same-person regions may fail to match and different-person regions may match.
- Global and fixed local representations assume similar poses and human spatial distributions within detected bounding boxes.Local representations commonly use horizontal stripes or grid cells.
- The proposed representation detects discriminative body regions, computes features over those parts, and aggregates similarities between corresponding parts.
- The method jointly learns body-part extraction and representation computation end-to-end without human body-part labels.
- Empirical results report competitive or superior performance on Market-1501, CUHK03, CUHK01, and VIPeR.
2. Related Work
Prior person re-identification work develops separate representations and matching mechanisms, increasingly using deep networks to learn features and similarities jointly. This paper instead focuses on a human body part-aligned representation learned from person similarity information without part annotations.
- Person re-identification research addresses representation and matching through separate or joint solutions.
- Hand-crafted representations and metric-learning methods target illumination, viewpoint, pose variation, and matching under other image differences.
- Deep solutions typically combine feature-extraction and matching subnetworks, using global features or features from stripes and grid cells.
- The proposed approach learns human body parts for matching from same-person and different-person similarity information, without labeled part or pose annotations.
- The method uses an attention-inspired human body part estimation scheme and reports better empirical performance than a related attention-model-and-LSTM approach.
3. Our Approach
The approach learns a part-aligned representation by detecting discriminative human regions, extracting features over them, and optimizing the resulting representation with triplet loss. It replaces fixed spatial partitioning with body-part partitioning to improve alignment under pose and spatial-distribution changes.
- Problem formulation: Person re-identification is formulated as ranking gallery images by distance to a probe, using positive same-person and negative different-person image pairs.The training set consists of triplets, and the margin requires negative pairs to be farther apart than positive pairs.
- Part-aligned representation: The representation extractor combines a fully convolutional network with a part net that detects aligned human regions instead of fixed grids or horizontal stripes.Each part-net branch receives the FCN feature map and detects a discriminative body region.
- Part-aligned representation: Each branch weights the image feature map with a learned part map, average-pools the weighted features, reduces their dimension, and concatenates the part features into the human representation.The final concatenated representation is L2-normalized before matching.
- Optimization: Network parameters are learned by minimizing triplet losses, with gradients reformulated so stochastic-gradient updates can use mini-batches of samples rather than explicitly drawing triplets.This sampling scheme reduces the computation associated with directly drawing triplets.
- Implementation details: The implementation uses a GoogLeNet-derived FCN, 400-image mini-batches, and reports 0.005 second per image for extracting the part-aligned representation at test time.Training uses SGD with ImageNet-pretrained initialization and runs for 50K iterations on a K40 GPU.
- Discussion: Learned parts are generally aligned across same-person image pairs, including cases with substantially different spatial distributions, while body-part partitioning outperforms stripe and grid alternatives.Separate part segmentation performs poorly because its parts are not learned directly for person re-identification.
4. Experiments
Experiments evaluate the approach on standard re-identification datasets using CMC curves and mAP, and compare part alignment with alternative representations and state-of-the-art methods. Results generally favor the proposed representation, while performance remains sensitive to dataset size and person-box quality.
- Datasets and evaluation: Experiments use Market-1501, CUHK03, CUHK01, and VIPeR, with single-query CMC evaluation and mAP reported on Market-1501.Market-1501 contains 1,501 identities, while CUHK03, CUHK01, and VIPeR provide additional cross-camera settings.
- Part configuration: Eight parts improve CUHK03 rank-1 performance over fewer parts, after which the scores become stable.Validation compares K = 1, 2, 4, 8, and 12 parts; scores at ranks 5, 10, and 20 are generally close.
- Baseline comparison: The part-aligned approach outperforms fully connected and average-pooling baselines on Market-1501 and CUHK03.The fully connected baseline has image-independent spatial weighting, whereas average pooling ignores spatial information; the proposed method adapts body-region differentiation per image.
- Backbone robustness: The approach remains better across AlexNet, VGGNet, and GoogLeNet feature-map extraction networks, except on CUHK01 with 100 test IDs.This supports using the part net with multiple feature extraction backbones, while identifying one dataset-specific exception.
- State-of-the-art comparison: Against competitive methods, the approach reports a 2.35 gain in rank-1 and 9.5 gain in mAP over PIE without KISSME.It also reports gains of 1.67 in rank-1 and 7.4 in mAP over PIE with KISSME, and a 4.5 rank-1 gain over PIE on the detected case.
- Scope and limitations: Performance is best with manually labeled boxes, and the authors identify person position in the box as influencing part-extraction quality.The paper suggests more supervision or larger datasets for learning a more robust part extractor; it also reports difficulty training deep networks from small data.
5. Conclusions
The paper presents a learned part-aligned representation that addresses body misalignment by partitioning human bodies into useful regions rather than fixed image grids or strips.
- The approach learns useful human body parts for person re-identification without human-part supervision.
- It jointly learns body-part partitioning and representation computation in a deep neural network inspired by attention models.
- Part-based representation is intended to improve robustness to pose changes and varying human spatial distributions within bounding boxes.