Source-linked AI summary
Deep Transfer Learning for Person Re-identification
Mengyue Geng, Yaowei Wang, Tao Xiang, Yonghong Tian
TL;DR
Person re-identification must learn deep models from small datasets with few or no labels. The paper proposes transfer-learning architectures, fine-tuning, and co-training, concluding that these models enable effective Re-ID without labelled data.
Problem
Deep Re-ID models achieve limited success because person re-identification datasets contain few labelled identities, making large-scale training difficult.
Method
The paper combines ImageNet-suitable network design with classification and verification losses, distinct dropout strategies, two-step fine-tuning, and co-training for unlabelled target data.
Results
The experiments validate the combined deep architecture and losses, while the co-training model performs effective Re-ID without any labelled data.
Takeaways & Limitations
Deep transfer learning can address person Re-ID data sparsity, including settings where target data are unlabelled.
Takeaways & Limitations
When more than one source dataset is used, the unsupervised transfer setup merges them into one.
Abstract
from arXiv · showhide
Person re-identification (Re-ID) poses a unique challenge to deep learning: how to learn a deep model with millions of parameters on a small training set of few or no labels. In this paper, a number of deep transfer learning models are proposed to address the data sparsity problem. First, a deep network architecture is designed which differs from existing deep Re-ID models in that (a) it is more suitable for transferring representations learned from large image classification datasets, and (b) classification loss and verification loss are combined, each of which adopts a different dropout strategy. Second, a two-stepped fine-tuning strategy is developed to transfer knowledge from auxiliary datasets. Third, given an unlabelled Re-ID dataset, a novel unsupervised deep transfer learning model is developed based on co-training. The proposed models outperform the state-of-the-art deep Re-ID models by large margins: we achieve Rank-1 accuracy of 85.4\%, 83.7\% and 56.3\% on CUHK03, Market1501, and VIPeR respectively, whilst on VIPeR, our unsupervised model (45.1\%) beats most supervised models.
1. Introduction
Person re-identification remains difficult because datasets are small and often sparsely labelled, limiting deep models and complicating transfer from ImageNet. The paper addresses this with an ImageNet-suitable architecture, combined classification and verification losses, two-step fine-tuning, and co-training for unlabelled targets.
- Re-ID matches people across non-overlapping camera views, but remains unsolved and deep models have achieved only limited success.
- Small labelled datasets constrain deep Re-ID, because collecting matching person images across camera networks is notoriously difficult.Published datasets include only 1,360 unique identities in CUHK03 and 1,501 in Market1501.
- ImageNet transfer is challenging because object categorisation, person-instance verification, image aspect ratios, and resolutions differ substantially.Existing Re-ID architectures are therefore often unsuitable for transferring ImageNet representations.
- The proposed architecture uses an ImageNet-optimised GoogleNet base with combined identity-classification and person-verification losses.The combination is intended to bridge the task discrepancy between object categorisation and person-instance verification.
- The method adds distinct dropout strategies for the two losses and a two-stepped fine-tuning procedure for small target datasets.The paper reports that two-stepped fine-tuning is more effective than conventional one-stepped fine-tuning.
- For unlabelled Re-ID data, the paper proposes co-training that alternates graph-regularised dictionary learning with soft-label self-training.The dictionary model supplies soft labels to the deep model, while graph regularisation is provided in the opposite direction.
2. Related Work
Prior deep Re-ID work mainly uses task-specific architectures and metric-learning losses, which limits direct use of ImageNet representations. The paper instead develops transfer-oriented architecture, fine-tuning, and co-training strategies to address these gaps.
- Deep Re-ID model: Existing deep Re-ID models primarily formulate matching through pairwise verification, triplet ranking, or both, using Siamese networks with multiple branches.
- Deep Re-ID model: Recent Re-ID models use smaller filters, fewer pooling layers, and nonsquare-input designs that can be trained from scratch on Re-ID datasets.
- Deep Re-ID model: These simplified architectures and the absence of combined classification-verification losses prevent existing models from exploiting ImageNet’s transferable representations.
- Deep transfer learning: Conventional one-step fine-tuning is inadequate for the large ImageNet-to-Re-ID task discrepancy, motivating the proposed two-stepped strategy.
- Deep transfer learning: The co-training method targets model drift in self-training and is reported to outperform alternative unsupervised models and most supervised models.The paper attributes the limitation of distribution-alignment methods to significant domain gaps between Re-ID datasets.
3. Deep Re-ID Model
The proposed deep Re-ID model uses a two-branch Siamese architecture combining identity classification and pairwise verification, with GoogLeNet chosen to support ImageNet transfer. Different dropout strategies address the distinct requirements of classification and verification.
- 3.1. Network Architecture: The model is a two-branch Siamese network that learns identity-discriminative appearance representations from paired person images.It contains a base network, classification subnet, verification subnet, and associated training objectives.
- 3.1. Network Architecture: The model combines an identity classification loss with a pairwise verification loss to distinguish people and match images of the same person.The classification objective supports auxiliary ImageNet pretraining, while verification targets pairwise identity comparison.
- 3.1. Network Architecture: GoogLeNet serves as the base network because it is competitive on ImageNet and supports transfer of generalisable representations.The architecture deliberately uses an established ImageNet-oriented network rather than a bespoke simplified Re-ID design.
- 3.1. Network Architecture: Pairwise-consistent dropout applies the same mask to compared images, avoiding appearance differences caused by independently sampled masks.The authors report a 3% improvement over random dropout for the verification loss.
- 3.1. Network Architecture: The large network is not effectively trainable from scratch on existing Re-ID datasets, motivating auxiliary-data transfer learning.The testing stage uses the learned model to match a probe image against gallery images.
4. Deep Transfer Learning for Re-ID
The paper transfers knowledge from ImageNet through staged fine-tuning for labelled targets and combines self-training with graph-regularised subspace learning for unlabelled targets. The unsupervised procedure iteratively refines representations and soft labels using cross-view relationships.
- 4. Deep Transfer Learning for Re-ID: Transfer learning is supervised when target identities are labelled and unsupervised when the target Re-ID dataset is unlabelled.The supervised setting distinguishes large targets with more than 1,000 identities from small targets with fewer than 1,000.
- Staged transfer learning: ImageNet transfer uses one stage for large Re-ID datasets and two stages, ImageNet → large Re-ID datasets → small Re-ID dataset, for small targets.Each stage uses a two-stepped fine-tuning strategy tailored to the combined classification and verification losses.
- Staged transfer learning: For a source-to-target transfer, the source identity classifier is replaced because source and target identities do not overlap.The replacement softmax layer is randomly initialised with one node per target identity.
- 4.2. Unsupervised Transfer Learning: For unlabelled targets, self-training assigns pseudo-labels from cross-view nearest neighbours, then repeatedly updates the network and regenerates soft labels.The pseudo-labels are noisy because visual nearest neighbours do not always share identity; the process typically converges after 2-3 iterations.
- 4.2. Unsupervised Transfer Learning: The co-training model complements self-training with graph-regularised subspace learning over base-network features.Its graph connects nearest-neighbour points from different camera views, and the learned subspace generates soft labels for target data.
5. Experiments
Experiments evaluate the proposed transfer-learning models on large and small Re-ID datasets under supervised and unsupervised settings. Results, ablations, and qualitative analyses examine transfer stages, losses, dropout, base networks, and co-training.
- Supervised transfer learning: The model significantly outperforms state-of-the-art methods on CUHK03 and Market1501.The reported gaps are 10.1% and 16.0% on CUHK03, and 26.0% in Market1501 mAP under single-query evaluation.
- Supervised transfer learning: The proposed model achieves the best results on VIPeR, PRID, and CUHK01 among the compared methods.Improvements are around 3% on VIPeR and PRID, with a larger gap on CUHK01.
- Ablation studies: The combined ID classification and pairwise verification losses support transfer from ImageNet classification to Re-ID verification.The experiments also compare alternative loss selections on Market1501.
- Supervised transfer learning: Two-stepped fine-tuning contributes about 8.7% at Rank 1 on VIPeR compared with one-stepped fine-tuning.Small datasets use two-staged transfer learning from ImageNet through CUHK03 and Market1501 before target fine-tuning.
- Unsupervised transfer learning: The unsupervised co-training model beats existing hand-crafted methods by large margins and is competitive with supervised results.It alternates a soft-label self-training deep model with a graph-regularised subspace-learning model; combining them improves performance by 2–3%.
- Qualitative results: ImageNet pretraining produces features that localise body parts more accurately than training without ImageNet pretraining.The qualitative analysis links this to more accurate delineation of persons and parts such as the head, torso, and arms.
6. Conclusion
The paper proposes deep transfer learning models for person re-identification with small datasets, including a co-training model that performs Re-ID without labelled data.
- The proposed models address person Re-ID with small datasets through deep transfer learning.
- Combining a deep base network with classification and verification losses is key to transferring representations from large image classification datasets.
- A co-training-based deep unsupervised transfer learning model performs effective Re-ID without labelled data.