Source-linked AI summary
Improving Person Re-identification by Attribute and Identity Learning
Yutian Lin, Liang Zheng, Zhedong Zheng, Yu Wu, Zhilan Hu, Chenggang Yan, Yi Yang
TL;DR
Existing re-ID methods mainly use coarse identity labels, whereas attribute labels provide detailed local descriptions that may improve pedestrian representations. The paper proposes APR, which jointly learns re-ID and attribute recognition while re-weighting correlated attribute predictions. APR achieves competitive re-ID performance on two benchmarks, accelerates Market-1501 retrieval by ten times with a 2.92% accuracy drop, and also improves attribute recognition.
Problem
Existing large-scale re-ID datasets contain identity labels but lack complementary attribute annotations that describe pedestrians in greater detail.
Method
APR jointly learns a re-ID embedding and pedestrian attributes from identity and attribute labels, using an Attribute Re-weighting Module to exploit attribute dependencies and correlations.
Results
APR achieves competitive re-ID performance on two large-scale benchmarks, accelerates Market-1501 retrieval by ten times with a 2.92% accuracy drop, and improves attribute recognition over baselines.
Takeaways & Limitations
Combining global identity information with local attribute descriptions supports discriminative pedestrian representations for both re-ID and attribute recognition.
Takeaways & Limitations
APR's multi-task optimization can reduce recognition rates for some attributes, including hair and B.pack in Market-1501, despite improving re-ID performance.
Abstract
from arXiv · showhide
Person re-identification (re-ID) and attribute recognition share a common target at learning pedestrian descriptions. Their difference consists in the granularity. Most existing re-ID methods only take identity labels of pedestrians into consideration. However, we find the attributes, containing detailed local descriptions, are beneficial in allowing the re-ID model to learn more discriminative feature representations. In this paper, based on the complementarity of attribute labels and ID labels, we propose an attribute-person recognition (APR) network, a multi-task network which learns a re-ID embedding and at the same time predicts pedestrian attributes. We manually annotate attribute labels for two large-scale re-ID datasets, and systematically investigate how person re-ID and attribute recognition benefit from each other. In addition, we re-weight the attribute predictions considering the dependencies and correlations among the attributes. The experimental results on two large-scale re-ID benchmarks demonstrate that by learning a more discriminative representation, APR achieves competitive re-ID performance compared with the state-of-the-art methods. We use APR to speed up the retrieval process by ten times with a minor accuracy drop of 2.92% on Market-1501. Besides, we also apply APR on the attribute recognition task and demonstrate improvement over the baselines.
1. Introduction
The paper argues that identity labels provide coarse global supervision, while attribute labels add detailed local descriptions that can improve person re-ID representations. It introduces APR, attribute re-weighting, annotated datasets, and attribute-based retrieval acceleration, reporting competitive re-ID accuracy and improved attribute recognition.
- Existing large-scale re-ID datasets primarily provide identity labels, motivating complementary attribute annotations for more detailed pedestrian descriptions.
- Attribute labels can bring visually similar identities closer and visually different identities farther apart than identity-only supervision.The paper presents this as a more natural organization of pedestrians in the target space.
- APR jointly learns a discriminative CNN embedding for person re-identification and attribute recognition using identity and attribute labels.
- The Attribute Re-weighting Module corrects attribute predictions using dependencies and correlations among attributes.The paper gives co-occurring attributes such as skirt and handbag as motivation for re-weighting gender predictions.
- 10 times smaller gallery size with a 2.92% accuracy drop is reported for attribute-accelerated retrieval on Market-1501.The process filters gallery images whose reliable attribute predictions differ from the query.
- APR achieves competitive re-ID accuracy on Market-1501 and DukeMTMC-reID and improves performance on attribute recognition.
2. Related Work
Prior work used attributes as auxiliary information, separate training signals, or transferred knowledge for person re-ID. The paper positions APR as a jointly learned approach that exploits mutual identity–attribute information and attribute correlations.
- The paper adopts a classification baseline and further exploits the mutual benefit between identity labels and attribute labels in one framework.
- Earlier methods integrated attributes with low-level descriptors, SVMs, metric learning, camera correlations, or dictionary learning for re-identification.
- A coarse-to-fine framework trained separate networks for person detection, gender prediction, and re-ID, potentially overlooking identity–attribute complementarity.
- Representative attributes include short sleeve, backpack, dress, and blue lower-body clothing, with positive and negative examples shown.
- Other approaches pre-trained or fine-tuned networks on independent attribute datasets before applying identity-based re-ID training.
3. Attribute Annotation
The authors manually annotate identity-level pedestrian attributes for Market-1501 and DukeMTMC-reID, using dataset-specific attribute sets and examining their distributions and correlations.
- Dataset-specific annotation: Market-1501 and DukeMTMC-reID receive manually annotated attribute labels, with distinct attribute sets reflecting their differing clothing characteristics.The datasets differ in seasons and clothing, motivating separate annotation schemes.
- Market-1501: Market-1501 is annotated with 27 attributes spanning demographics, clothing, accessories, clothing colors, and age.The labels include gender, hair and clothing lengths, clothing type, bags, upper- and lower-body colors, and age.
- Attribute distributions: Figure 3 reports positive-identity counts for most attributes and color distributions for upper- and lower-body clothing on both datasets.The left plots show positive IDs, while the middle and right pie charts show upper- and lower-body clothing colors.
- DukeMTMC-reID: DukeMTMC-reID is annotated with 23 attributes covering demographics, footwear, accessories, clothing length, and upper- and lower-body colors.Its labels include shoe type and color, bags, hats, clothing length, and clothing colors.
- Annotation protocol: Attributes are annotated at the identity level, so the same attribute label is retained across images of one person even when it is not clearly visible.The paper illustrates this with a backpack that is obscured in one image of an identity.
- Attribute correlations: Attribute correlation is defined by the possibility that two attributes co-occur on a person, with darker grids representing higher correlations in Figure 4.Figure 4 displays correlations for selected representative attribute pairs.
4. The Proposed Method
APR jointly learns attribute recognition and identity classification, using re-weighted local attribute predictions alongside global image features for person re-identification. It also supports attribute-based gallery filtering to accelerate retrieval.
- Attribute Re-weighting Module: ARM learns attribute confidence weights from attribute correlations and applies them to the original prediction scores before identity classification.The module uses trainable parameters and a Sigmoid gating mechanism.
- APR network: APR jointly optimizes attribute recognition and identity classification on combined identity- and attribute-labeled data.The shared image representation supports both attribute classifiers and identity prediction.
- APR network: The identity classifier uses re-weighted attribute predictions concatenated with the global image feature as a local-global representation.Attribute predictions are re-weighted before identity classification.
- Optimization: The overall objective balances identity classification and attribute recognition losses with hyper-parameter λ.λ controls the relative contribution of the two task losses.
- Attribute acceleration process: Attributes can filter gallery images whose reliable predictions disagree with the query, enabling retrieval acceleration.An attribute is treated as reliable when its confidence score exceeds threshold τ.
- Attribute acceleration process: Over ten times retrieval speedup was achieved on Market-1501 with a 2.92% accuracy drop at threshold 0.7.The approach is intended for applications where retrieval speed is the main focus.
5.1. Datasets and Evaluation Protocol
Experiments evaluate person re-identification on Market-1501 and DukeMTMC-reID, and attribute recognition on PETA, using task-specific annotations and metrics.
- Datasets: Experiments use Market-1501, DukeMTMC-reID, and PETA for re-identification and attribute recognition evaluation.Market-1501 and DukeMTMC-reID are re-ID datasets; PETA is an attribute recognition dataset.
- Datasets: Market-1501 contains 19,732 training images for 751 identities and 13,328 testing images for 750 identities, with 27 attributes per image.A 651/100 identity split is used to validate λ before the normal 751/750 split.
- Datasets: DukeMTMC-reID contains 16,522 training images for 702 identities and 19,889 testing images for 702 identities, with 23 labels per image.The dataset is a subset of DukeMTMC.
- Datasets: PETA provides 19,000 images with 61 binary and 4 multi-class attributes, of which 35 attributes are used in the experiments.The paper re-splits PETA for the re-ID task because most identities have few training images.
- Evaluation protocol: Re-ID is evaluated with CMC and mAP, while attribute recognition is evaluated using classification accuracy for each attribute.Gallery images serve as the testing set for attribute prediction.
5.2. Implementation details
Implementation uses ImageNet-initialized ResNet-50 or CaffeNet backbones with task-specific feature concatenation, training schedules, and image augmentation.
- Backbones: APR uses ImageNet-pretrained ResNet-50 and CaffeNet CNN backbones.The two backbones are used as alternative CNN feature extractors.
- Feature construction: With ResNet-50 on Market-1501, a 512-dimensional feature is concatenated with 27 attribute scores to form a 539-dimensional identity-classification feature.Attribute classifiers are attached after the pool5 layer.
- Training: ResNet-50 training runs for 60 epochs with batch size 32 and learning rate 0.01, reduced to 0.001 during the final 20 epochs.A fully connected layer, Batch Normalization, dropout, and ReLU follow pool5.
- Training: CaffeNet training runs for 110 epochs with batch size 128, using learning rate 0.1 for 100 epochs and 0.01 for the final ten epochs.Random cropping and horizontal flipping are applied during training.
5.3. Evaluation of Person Re-ID task
APR combines identity and attribute learning to improve person re-ID across multiple datasets, while attribute filtering accelerates retrieval with limited accuracy loss.
- Comparison with the state-of-the-art methods: APR achieves rank-1 = 87.04% and mAP = 66.89% on Market-1501, and rank-1 = 73.92% and mAP = 55.56% on DukeMTMC-reID.Using ResNet-50, APR compares favorably with state-of-the-art methods on both benchmarks.
- Comparison with the baselines: 6.88% and 37.28% are APR’s Market-1501 rank-1 improvements over Baseline 1 and Baseline 2, respectively, using ResNet-50.On DukeMTMC-reID, the corresponding improvements are 9.7% and 27.78%; PETA shows 4.15% and 14.65% improvements.
- Ablation Studies: 2.14% and 1.49% rank-1 decreases follow removal of bag types on Market-1501 and shoe color on DukeMTMC-reID, respectively.Most attributes are indispensable, whereas wearing a hat has a small negative impact on overall re-ID accuracy.
- Algorithm Analysis: 1.33% rank-1 and 0.30% mAP gains on Market-1501 result from the Attribute Re-weighting Module; gains also occur on DukeMTMC-reID and PETA.The improvements are consistent across all reported experiments.
- Algorithm Analysis: λ = 0.9 gives the best validation performance, with rank-1 accuracy 94.52% and mAP 88.03% on Market-1501.Identity classification becomes more important as λ increases, but performance declines at λ = 1, indicating attributes remain effective.
- Robustness and retrieval acceleration: Nearly ten times faster retrieval leaves 8.68% of the gallery and causes only a 2.92% accuracy drop on Market-1501.With about 2,000 gallery images per query, testing time falls from 919.86s to 90.26s.
5.4. Evaluation of Attribute Recognition
APR improves attribute recognition across all evaluated datasets, although some individual attributes decline in recognition accuracy under the multi-task objective.
- Attribute recognition results: 0.26%, 0.08%, 0.2% and 1.58% are APR’s overall attribute-recognition improvements on Market-1501, DukeMTMC-reID, PETA and CUB 200 2011, respectively.The reported gains are measured against Baseline 2 (ARN).
- Attribute recognition results: APR achieves the best attribute-recognition result on PETA among the state-of-the-art methods.The evaluation covers attribute recognition on Market-1501, DukeMTMC-reID, PETA and CUB 200 2011.
- Limitations: Recognition rates decrease for some attributes, including hair and B.pack on Market-1501.The paper notes that these attributes can nevertheless be necessary for improving re-ID performance.
6. Conclusions and Future Work
The paper presents APR as a joint framework for learning re-ID embeddings and pedestrian attributes from complementary identity and attribute labels. It also reports retrieval acceleration and attribute-recognition gains, while identifying transferability, scalability, and attribute-based retrieval as future directions.
- Conclusions: APR jointly learns a re-ID embedding and pedestrian attributes, re-weighting attribute predictions according to attribute dependencies and correlations.The method integrates identity classification and attribute recognition under one framework.
- Conclusions: APR achieves competitive re-ID performance on two large-scale benchmarks and improves overall attribute-recognition precision.The paper additionally applies APR to accelerate retrieval on Market-1501.
- Future Work: Future work targets transferability and scalability of pedestrian attributes across datasets and attribute-based retrieval when a query image is unavailable.The proposed direction includes adapting an attribute model learned on Market-1501 to other pedestrian datasets.