Source-linked AI summary
Deep Attributes Driven Multi-Camera Person Re-identification
Chi Su, Shiliang Zhang, Junliang Xing, Wen Gao, Qi Tian
TL;DR
Person ReID is challenged by pose, viewpoint, and camera differences, creating a need for robust human representations. The paper proposes three-stage SSDAL training that refines attributes with person-ID data, and reports strong performance and generalization across four datasets. It also identifies cases where expanded or weakly supervised fine-tuning does not consistently improve results.
Problem
Person ReID requires representations robust to pose, viewpoint, and camera variations, but learning many human attributes is laborious because sufficient attribute-labeled data is difficult to obtain.
Method
SSDAL trains an attribute-labeled dCNN, fine-tunes it with person-ID labels and attributes triplet loss, then fine-tunes on original and predicted target attributes.
Results
Deep attributes show promising generalization and substantially outperform previous person ReID methods across four benchmark datasets, while fusion with LOMO reaches a CMC score of 45.3 at Rank-1.
Takeaways & Limitations
Deep attributes can support person ReID across different datasets without further target-dataset training and reduce dependence on local features.
Takeaways & Limitations
Stage1&3 does not constantly outperform the baseline, and Stage1&2 may generalize poorly because person-ID labels can be weak and fully connected layers can overfit.
Abstract
from arXiv · showhide
The visual appearance of a person is easily affected by many factors like pose variations, viewpoint changes and camera parameter differences. This makes person Re-Identification (ReID) among multiple cameras a very challenging task. This work is motivated to learn mid-level human attributes which are robust to such visual appearance variations. And we propose a semi-supervised attribute learning framework which progressively boosts the accuracy of attributes only using a limited number of labeled data. Specifically, this framework involves a three-stage training. A deep Convolutional Neural Network (dCNN) is first trained on an independent dataset labeled with attributes. Then it is fine-tuned on another dataset only labeled with person IDs using our defined triplet loss. Finally, the updated dCNN predicts attribute labels for the target dataset, which is combined with the independent dataset for the final round of fine-tuning. The predicted attributes, namely \emph{deep attributes} exhibit superior generalization ability across different datasets. By directly using the deep attributes with simple Cosine distance, we have obtained surprisingly good accuracy on four person ReID datasets. Experiments also show that a simple metric learning modular further boosts our method, making it significantly outperform many recent works.
1 Introduction
Person ReID must handle substantial appearance changes across cameras, motivating robust mid-level human attributes and a semi-supervised learning framework. SSDAL progressively trains attribute detectors and achieves strong cross-dataset performance without relying on local visual features.
- Viewpoint, illumination, camera, and pose variations make identifying the same person across cameras challenging.
- Human attributes provide mid-level semantic representations that are more consistent across appearance variations than low-level visual features.
- The approach aims to learn many human attributes from limited labeled attribute data while improving robustness and efficiency by removing dependence on local features.
- SSDAL uses three stages: attribute-supervised dCNN training, person-ID fine-tuning with attributes triplet loss, and final fine-tuning on original plus predicted labels.The stages progressively produce and refine attribute labels for target data.
- The method evaluates predicted attributes on four person ReID datasets without combining them with local visual features.
- Deep attributes significantly outperform many recent methods that combine attributes with local features.
2 Related Work
Prior ReID work learns visual features or distance metrics, often with dataset- or camera-specific dependence. This paper instead targets general camera-independent mid-level representations based on human attributes.
- Deep learning has been applied to attribute learning and person ReID, including feature learning, distance metrics, and pairwise identity classification.
- Existing deep ReID approaches are designed for particular datasets and depend on their camera settings.
- The paper seeks camera-independent mid-level representations that provide greater flexibility across datasets.
- SSDAL uses attributes triplet loss to fine-tune attribute detectors, rather than learning a distance metric among low-level features.
3 Proposed Approach
The proposed SSDAL framework learns discriminative human attributes from limited labeled attribute data by progressively combining attribute supervision, person-ID supervision, and predicted labels. Its final detector produces sparse binary attribute features for person ReID.
- Framework: The framework uses a dCNN to predict K-dimensional binary attribute labels for person images.Each attribute is represented by a binary indicator, and the detector maps an input image to its attribute label.
- Framework: SSDAL trains an attribute detector through three stages: independent attribute supervision, person-ID fine-tuning, and final fine-tuning on combined labeled data.The updated network predicts attributes for the person-ID dataset before it is merged with the independent attribute-labeled dataset.
- dCNN Fine-tuning with Attributes Triplet Loss: Person-ID fine-tuning constructs anchor-positive-negative triplets and minimizes distance for same-person attributes while maximizing distance for different-person attributes.The positive sample shares the anchor's identity, whereas the negative sample has a different identity.
- dCNN Fine-tuning with Attributes Triplet Loss: Regularization prevents fine-tuning from producing meaningless attributes or overfitting the person-ID dataset.The modified loss constrains attribute changes while preserving similar attributes for images of the same person.
- Final Attribute Detector: The final detector selects attributes above a confidence threshold to form a sparse binary feature vector, whose distances are measured with Cosine distance for ReID.The confidence threshold is experimentally set to 0.
4 Experiments
Experiments evaluate attribute prediction and person ReID across three training stages, two-camera datasets, and a multi-camera dataset. The results show improved accuracy, strong cross-dataset performance, and further gains from metric learning or feature fusion.
- Training setup: The first stage trains an AlexNet-based dCNN on PETA attributes, while the second stage uses MOT person IDs and attribute triplet loss for fine-tuning.PETA provides 105 binary attributes after expanding its multiclass labels.
- Evaluation setup: The evaluation tests predicted attributes on VIPeR, PRID, and GRID, while ReID performance is also assessed on VIPeR, PRID, GRID, and Market.Target datasets are excluded from dCNN training when used for testing.
- Attribute prediction: Stage1&3 does not consistently outperform the baseline because its additional training data is labeled by the baseline network and adds no new cues.The expanded set is therefore not guaranteed to improve fine-tuning.
- Attribute prediction: Stage1&2 also does not consistently outperform the baseline, potentially because person-ID labels are weak and fully connected layers can overfit other datasets.The reported concern is reduced generalization beyond the fine-tuning dataset.
- Attribute prediction: SSDAL improves baseline attribute accuracy by 1.2% on average across three datasets, indicating progressively more robust semantic attributes.The final model is obtained after the three-stage training framework.
- Two-camera ReID: Fine-tuning with attribute triplet loss improves rank-1 accuracy by 3.4% on VIPeR, 1.4% on PRID, and 5.3% on GRID.These gains support progressively adding information during training.
- Two-camera ReID: SSDAL surpasses existing algorithms on PRID and GRID, while SSDAL + XQDA achieves the best rank-1 accuracy across all three two-camera datasets.The comparison also reports consistent advantages across various ranks on PRID and GRID.
- Multi-camera ReID: On Market, the approach outperforms all compared methods in both single-query and multiple-query settings; multiple-query mAP rises from 18.5% to 25.8%.This is a 7.3% absolute improvement in the multiple-query scenario.
5 Conclusions and Future Work
The paper concludes that deeply learned human attributes and SSDAL provide robust person ReID performance across datasets. It also identifies one-time training and ID-labeled fine-tuning as practical advantages, while noting spatial attribute relationships as future work.
- SSDAL uses deeply learned human attribute features to address person ReID.
- Images with only person ID labels can train attribute detectors through the attributes triplet loss.
- Experiments on four benchmark datasets show robust attribute detection and substantially better performance than previous person ReID methods.
- The attribute-prediction dCNN requires no further target-dataset training and can be trained once for ReID across different datasets.
- Future work includes modeling the spatial locations and correlations of attributes to improve attribute-detection accuracy.