Source-linked AI summary
A Multi-task Deep Network for Person Re-identification
Weihua Chen, Xiaotang Chen, Jianguo Zhang, Kaiqi Huang
TL;DR
Person ReID must identify the same person across scenes despite appearance variation, but classification and ranking losses each have limitations and target datasets are often small. The paper jointly optimizes both losses in MTDnet and extends it with cross-domain training using an auxiliary dataset. The approach outperforms most existing methods across representative ReID datasets.
Problem
Person ReID must identify people across scenes despite appearance variation, while existing approaches separately use classification or ranking and often lack sufficient training data.
Method
MTDnet jointly optimizes classification and ranking losses on different network layers, with a cross-domain architecture using auxiliary data for small target datasets.
Results
The proposed approach outperforms most existing person ReID algorithms on CUHK03, CUHK01, VIPeR, iLIDS, and PRID2011.
Takeaways & Limitations
Jointly optimizing the complementary classification and ranking tasks, together with cross-domain training, provides a strong approach for person ReID across large and small datasets.
Takeaways & Limitations
Fixed embedding metrics such as Euclidean distance can make learning a suitable feature representation difficult.
Abstract
from arXiv · showhide
Person re-identification (ReID) focuses on identifying people across different scenes in video surveillance, which is usually formulated as a binary classification task or a ranking task in current person ReID approaches. In this paper, we take both tasks into account and propose a multi-task deep network (MTDnet) that makes use of their own advantages and jointly optimize the two tasks simultaneously for person ReID. To the best of our knowledge, we are the first to integrate both tasks in one network to solve the person ReID. We show that our proposed architecture significantly boosts the performance. Furthermore, deep architecture in general requires a sufficient dataset for training, which is usually not met in person ReID. To cope with this situation, we further extend the MTDnet and propose a cross-domain architecture that is capable of using an auxiliary set to assist training on small target sets. In the experiments, our approach outperforms most of existing person ReID algorithms on representative datasets including CUHK03, CUHK01, VIPeR, iLIDS and PRID2011, which clearly demonstrates the effectiveness of the proposed approach.
Introduction
Person ReID must handle appearance variation while balancing classification and ranking objectives. The paper proposes jointly optimizing both tasks and using cross-domain training when target datasets are small.
- Motivation: Person ReID identifies people across scenes despite changes in pose, illumination, and camera view.Applications include inter-camera pedestrian tracking and human retrieval.
- Classification and ranking challenges: Binary classification can favor lower misclassification at the expense of correct per-query ranking.Case 1 yields correct rank-1 matches but is less attractive to classification loss than Case 2.
- Classification and ranking challenges: Ranking losses can prioritize visually similar mismatches over true positives with less similar appearance.This makes ranking-based approaches sensitive to their similarity measurements.
- Proposed approach: The proposed network jointly optimizes classification and ranking losses on different layers.Ranking enforces relative distances, while classification learns discriminative pair features using joint feature maps.
- Cross-domain training: A cross-domain architecture uses auxiliary data to improve joint feature maps when target datasets have few samples.The auxiliary data cannot be directly transferred because camera-view variations differ across datasets.
- Evaluation: The method reports superior performance over most state-of-the-art methods across five person ReID datasets.The evaluation covers CUHK03, CUHK01, VIPeR, iLIDS, and PRID2011.
Related work
Earlier ReID methods separately addressed feature extraction or similarity measurement, while deep approaches commonly used either classification or ranking. This paper combines both tasks and addresses training on small datasets through cross-domain learning.
- Prior approaches: Traditional ReID systems often separate feature extraction from similarity measurement, producing suboptimal end-to-end optimization.They use descriptors such as color histograms, local binary patterns, and Gabor features.
- Prior approaches: Deep ReID methods commonly formulate the task as binary classification, while others apply ranking losses such as triplet loss.These approaches emphasize different aspects of representation learning and similarity ordering.
- Small-data setting: Small datasets such as VIPeR and PRID2011 contain only two images per person, limiting the effectiveness of multi-class classification.Some prior work addresses this by combining multiple datasets as training data.
- Paper contribution: Unlike prior work that combines losses at the score level or trains them separately, this network jointly optimizes classification and ranking losses.The two tasks are trained simultaneously within one network.
- Paper contribution: The cross-domain architecture addresses deep-network training on small ReID datasets by learning across ReID domains.The paper evaluates the approach on five datasets and reports superior performance over most state-of-the-art methods.
The proposed network
The proposed network combines ranking and classification through complementary layers, then extends this design with cross-domain transfer for small target datasets.
- Multi-task architecture: The architecture assigns ranking to early layers for global features and classification to deeper layers for identifiable local features.The first two convolutional layers are optimized by triplet loss, while classification loss trains all five convolutional layers.
- Multi-task architecture: The ranking branch uses triplet inputs and shared features to preserve relative distances among positive and negative images.It processes three images per triplet and minimizes a FaceNet-style triplet loss.
- Multi-task architecture: The classification branch forms positive and negative image pairs, learns joint feature maps, and predicts whether each pair shows the same person.Positive pairs concatenate images of the same person, while negative pairs combine identities across camera views.
- Cross-domain architecture: The cross-domain extension uses contrastive learning to align joint feature maps for same-class pairs across source and target datasets.Pairs with matching labels are pulled closer, while pairs with different labels are pushed farther.
- Cross-domain architecture: The cross-domain training jointly uses re-identification losses and contrastive loss, retaining only the target model for testing.The source dataset provides auxiliary feature-map information for fine-tuning the smaller target dataset.
Experiments
Experiments evaluate the multi-task and cross-domain architectures on representative person ReID datasets and compare them with existing methods. The proposed designs generally outperform single-loss, direct augmentation, and most competing approaches.
- Experimental setup: The experiments evaluate the proposed multi-task and cross-domain architectures and compare them with state-of-the-art methods.The study uses one large dataset and four small datasets, with CMC curves measuring ReID performance.
- Multi-task results: MTDnet performs better than its ranking-loss testing variant, indicating that deeper classification-trained layers improve ReID performance.The comparison is between MTDnet and MTDtrp using the same multi-task model.
- Experimental setup: Table 1 reports CMC performance for existing methods and the proposed architectures across five representative datasets.Bold entries indicate the best performance.
- Cross-domain results: The cross-domain architecture almost always improves over MTDnet on four small datasets and outperforms direct source-target data augmentation.CUHK03 is used as the source domain, while each small dataset is treated as a target domain.
- Comparison with prior methods: Under rank-1 accuracy, MTDnet outperforms all existing algorithms on CUHK03, CUHK01, and PRID2011.ImpTrpLoss achieves the best rank-1 performance on VIPeR and iLIDS, where the proposed results are comparable.
Conclusion
The proposed multi-task network integrates classification and ranking tasks for person ReID, while a cross-domain extension addresses small target datasets. Across large and small datasets, the approach outperformed almost all compared state-of-the-art methods.
- The multi-task network integrates classification and ranking tasks in one person ReID network.
- A cross-domain architecture fine-tunes joint feature maps for training with small target datasets.
- The proposed network outperformed almost all compared state-of-the-art methods on both large and small datasets.