PaperlayerView exact revision on arXiv

Source-linked AI summary

Unsupervised Pre-training for Person Re-identification

Dengpan Fu, Dongdong Chen, Jianmin Bao, Hao Yang, Lu Yuan, Lei Zhang, Houqiang Li, Dong Chen

arXiv:2012.03753v2cs.CV

TL;DR

Person Re-ID datasets are limited because collecting and annotating large, diverse datasets is difficult and expensive. This paper introduces LUPerson and studies unsupervised pre-training on it, reporting improved performance across supervised and unsupervised Re-ID, with larger gains in small-scale or few-shot settings.

  • Problem

    Collecting and annotating the large, diverse datasets needed for person Re-ID is difficult and expensive, leaving existing datasets limited in scale.

    Source: §3 · p. 3

  • Method

    The authors build LUPerson, a 4M-image unlabeled dataset, and study unsupervised contrastive pre-training choices for Re-ID, including augmentation and temperature.

    Source: §1 · pp. 1–2

  • Results

    Pre-training improves supervised and unsupervised Re-ID performance across datasets; on Market1501, it adds 3.5% mAP over the MGN baseline.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    The reported gains are larger with small-scale training samples or few-shot Re-ID datasets.

    Source: §1 · p. 2, §6 · p. 8

Abstract

from arXiv · show

In this paper, we present a large scale unlabeled person re-identification (Re-ID) dataset "LUPerson" and make the first attempt of performing unsupervised pre-training for improving the generalization ability of the learned person Re-ID feature representation. This is to address the problem that all existing person Re-ID datasets are all of limited scale due to the costly effort required for data annotation. Previous research tries to leverage models pre-trained on ImageNet to mitigate the shortage of person Re-ID data but suffers from the large domain gap between ImageNet and person Re-ID data. LUPerson is an unlabeled dataset of 4M images of over 200K identities, which is 30X larger than the largest existing Re-ID dataset. It also covers a much diverse range of capturing environments (eg, camera settings, scenes, etc.). Based on this dataset, we systematically study the key factors for learning Re-ID features from two perspectives: data augmentation and contrastive loss. Unsupervised pre-training performed on this large-scale dataset effectively leads to a generic Re-ID feature that can benefit all existing person Re-ID methods. Using our pre-trained model in some basic frameworks, our methods achieve state-of-the-art results without bells and whistles on four widely used Re-ID datasets: CUHK03, Market1501, DukeMTMC, and MSMT17. Our results also show that the performance improvement is more significant on small-scale target datasets or under few-shot setting.

1. Introduction

The paper introduces LUPerson, a large unlabeled person Re-ID dataset, and uses it to study unsupervised pre-training as a way to learn Re-ID features. It reports that the resulting representation benefits both supervised and unsupervised Re-ID methods.

1. Introduction

  • Existing Re-ID datasets are limited in image count, identities, and capture environments, while ImageNet pre-training may be suboptimal because of its domain gap from person Re-ID data.The passage gives MSMT17 as the largest dataset by image count at 126K images and Airport as the largest by identities at 9,651.Source: §1 · p. 1
  • LUPerson contains 4M images of over 200K identities, extracted from 46K videos, and is 30× larger than MSMT; its varied capture environments support generic representation learning.The paper presents it as enabling the first unsupervised pre-training for person Re-ID.Source: §1 · pp. 1–2
  • The authors identify three Re-ID-specific factors: color distortion harms learning, RandomErasing helps prevent trivial contrastive solutions, and temperature balances discriminativity with hard-negative mining.They report that directly applying MoCo v2 does not work well for Re-ID.Source: §1 · p. 2
  • Against MGN, LUPerson pre-training gains 3.5% mAP on Market1501, 2.7% on DukeMTMC, and 2.0% on MSMT17; against BDB, it gains 2.9% on CUHK03, surpassing state-of-the-art methods.The improvement is more significant with small-scale training samples, and the pre-trained model also consistently improves unsupervised methods based on SpCL.Source: Figure 1 · p. 1, §1 · p. 2

2. Related Work

Prior work addresses supervised Re-ID, label-free target-domain Re-ID, and generic contrastive pre-training; this paper instead pre-trains Re-ID features for use across supervised and unsupervised methods.

Supervised Person Re-ID

  • Supervised Re-ID uses global classification, global triplet, or part-based features; MGN [38] combines global classification and local triplet losses, and the pre-trained model improves representative methods.Examples include IDE [45] and [33], hard-triplet learning [22], horizontal-strip classification [36], and part-aligned bilinear representations [34].Source: §2 · p. 2

Unsupervised Person Re-ID

  • BUC, MMT [14], MMCL [37], and SpCL [15] use clustering, refined pseudo-labels, multi-label learning, or reliable-cluster contrastive fine-tuning, whereas this work pre-trains transferable features.The resulting representation is intended to generalize to supervised or unsupervised Re-ID approaches.Source: §2 · p. 2

Unsupervised Representation Learning

  • Prior contrastive pre-training uses memory banks, queues [19] [9], augmentation/projection heads [6] [7], or negative-free BYOL [17]; MoCo v2 [8] is chosen, but direct Re-ID use performs poorly.The paper motivates studying which factors distinguish person Re-ID from generic visual representation learning.Source: §2 · pp. 2–3

3. LUPerson: Large-scale Re-ID Dataset

Because existing Re-ID datasets are limited in scale and diversity and are costly to annotate, the paper introduces unlabeled LUPerson to support generic Re-ID feature learning. It describes how the dataset was collected and compares its scale and capture diversity with existing datasets.

3. LUPerson: Large-scale Re-ID Dataset

  • LUPerson is an unlabeled Re-ID dataset with 4M images, over 200K identities, and 46K scenes, built to support generic feature learning.The paper motivates it by the difficulty and expense of dataset annotation and the limited scale and diversity of existing Re-ID datasets.Source: §3 · p. 3

3.1. Data Collection and Processing

  • The authors built LUPerson from YouTube street-view videos, filtering 50,534 videos and extracting 4,180,243 person images.They selected cities to broaden capture diversity, removed duplicated, short, static, and virtual-reality videos, and used YOLO-v5 detection and HRNet keypoints to validate person images. They estimated 219,848 persons by summing each video's maximum number of visible persons.Source: §3.1 · pp. 3–4

3.2. Comparison with Existing Datasets

  • Over 4M images and 200K identities make LUPerson 30× larger than MSMT17 [39]; it spans 46,260 scenes and diverse camera views.The dataset also includes static and dynamic views, varied recording devices and lighting, and ethnic and pose diversity.Source: §3.2 · p. 4

4. Unsupervised Pre-training for Person Re-ID

LUPerson-based unsupervised pre-training aims to improve person Re-ID feature generalization, but directly applying MoCoV2 [9] does not work well, motivating task-specific study of augmentations and contrastive temperature.

4. Unsupervised Pre-training for Person Re-ID

  • MoCoV2 [9] is a simple baseline, but direct application does not learn a good person Re-ID representation, so the authors study task-specific changes.They focus on data augmentation and temperature tuning in the contrastive loss.Source: §4 · p. 4

4.1. Contrastive Learning

  • MoCo [19] treats two augmentations of one image as a positive pair and other images as negatives, using a queue and a momentum-updated key encoder.The method encodes query and key samples with separate encoders, each comprising a feature extractor and projection head.Source: §4.1 · p. 5

4.2. Key Factors to Study

  • On CUHK03, disabling color jitter improved mAP and cmc1 by 0.6% each, while RandomErasing added about 0.8% mAP and performed better at pre-training strength 0.6 than the supervised optimum 0.4.The authors attribute the color-jitter result to Re-ID’s reliance on color cues such as clothing and bags; they conclude that pre-training should remove color jitter and use strong RandomErasing.Source: §4.2 · p. 5, Table 2 · p. 6, Table 3 · p. 6
  • On CUHK03, lowering τ from 0.2 to 0.07 raised mAP from 71.1 to 74.7, while ImageNet recognition accuracy fell from 66.2 to 62.9 [9].The authors suggest Re-ID’s finer-grained distinctions and smaller inter-class variation make a lower temperature more appropriate.Source: §4.2 · pp. 5–6, Table 4 · p. 6

4.3. Transferring Features

  • An extra batch-normalization layer after each use of the pre-trained feature recalibrates it, allowing existing methods to retain hyperparameters tuned for ImageNet-supervised pre-training.For MGN [38], the authors add three such layers, one before each head.Source: §4.3 · p. 6

5. Experiments

The experiments show that LUPerson pre-training improves supervised and unsupervised Re-ID, including in small-data settings, and yields state-of-the-art results on three benchmarks.

5.1. Implementation

  • MoCoV2 pre-training used ResNet50 by default for 200 epochs on 8 × V100 GPUs, with batch size 2,560 and initial learning rate 0.3.Evaluation used mAP and cmc1 on CUHK03, Market1501, DukeMTMC, and MSMT17.Source: §5.1 · p. 6

5.2. Improving Supervised Re-ID Methods

  • Replacing supervised ImageNet pre-training with LUPerson unsupervised pre-training improved mAP by more than 4.2%, 3.5%, 2.7%, and 2% on CUHK03, Market1501, DukeMTMC, and MSMT17, respectively.Unsupervised ImageNet pre-training was slightly better than supervised ImageNet in most cases, but remained below LUPerson pre-training.Source: §5.2 · p. 6, Table 5 · p. 7

5.3. Comparison on Small-scale and Few-shot

  • With only 1,170 Market1501 images from 75 identities, LUPerson-pre-trained MGN achieved 64.6 mAP, 11.5 mAP above the ImageNet-supervised counterpart.Across the tested 10%–100% training-set percentages, pre-training boosted MGN in most small-scale and few-shot cases; few-shot gains became somewhat saturated as images per identity decreased, possibly because hard triplet loss was weakened.Source: §5.3 · p. 7, Table 6 · p. 7

5.4. Pre-training Data Scale

  • Increasing LUPerson's pre-training scale strengthened the learned representation, with gains tending to saturate at larger scales.The study used 12.5%, 25%, 50%, and 100% of LUPerson; the authors suggest greater network capacity may be needed to exploit more data further.Source: §5.4 · p. 8, Table 7 · p. 8

5.5. Improving Unsupervised Re-ID Methods

  • SpCL with LUPerson pre-training gained 3.8 mAP on Market1501 and 2.2 mAP on DukeMTMC for USL, and 2.8% and 1.2% mAP for M→D and D→M UDA.The authors report that the USL results set a new state of the art for person Re-ID.Source: §5.5 · p. 8, Table 8 · p. 8

5.6. Comparison with State-of-the-Arts

  • LUPerson-pre-trained MGN achieved state-of-the-art results on Market1501, DukeMTMC, and MSMT17 without post-processing; on CUHK03, BDB gained 2.9 mAP and beat SONA.Using ResNet101 instead of ResNet50 further improved representation performance.Source: §5.6 · p. 8, Table 9 · p. 8

6. Conclusion

The paper introduces LUPerson and uses it for unsupervised pre-training of general person Re-ID features. Experiments show benefits for supervised and unsupervised approaches, with larger gains on small-scale or few-shot datasets.

6. Conclusion

  • The paper releases LUPerson and uses it in a first attempt at unsupervised pre-training for general person Re-ID features.The dataset is intended to address the limited scale and diversity of existing Re-ID datasets.Source: §6 · p. 8
  • Experiments demonstrate the pre-training model’s effectiveness and generalization for supervised and unsupervised Re-ID, with bigger gains on small-scale or few-shot datasets.Source: §6 · p. 8
  • The paper identifies incorporating video temporal information into pre-training and developing end-to-end unsupervised Re-ID that can beat supervised methods as future directions.Source: §6 · p. 8

Related papers

Loading 2012.03753v2…