PaperlayerView exact revision on arXiv

Source-linked AI summary

Unsupervised Person Re-identification: Clustering and Fine-tuning

Hehe Fan, Liang Zheng, Yi Yang

arXiv:1705.10444v2cs.CV

TL;DR

The paper addresses learning deep person re-identification features for unseen domains when target labels are absent or scarce. PUL alternates clustering and CNN fine-tuning while selecting reliable samples through a self-paced process, and experiments on three large-scale datasets report improved re-ID accuracy.

  • Problem

    Most deep re-ID methods rely on abundant labeled data from the same environment, while adapting to new environments has high annotation cost.

  • Method

    PUL transfers a CNN initialized on external labeled data by alternating k-means clustering, centroid-based reliable-sample selection, and CNN fine-tuning.

  • Results

    PUL noticeably improves re-ID accuracy in cross-dataset evaluation on three large-scale datasets.

  • Takeaways & Limitations

    Reliable sample selection is necessary for PUL, while progressively training on selected samples produces more discriminative CNN models.

Abstract

from arXiv · show

The superiority of deeply learned pedestrian representations has been reported in very recent literature of person re-identification (re-ID). In this paper, we consider the more pragmatic issue of learning a deep feature with no or only a few labels. We propose a progressive unsupervised learning (PUL) method to transfer pretrained deep representations to unseen domains. Our method is easy to implement and can be viewed as an effective baseline for unsupervised re-ID feature learning. Specifically, PUL iterates between 1) pedestrian clustering and 2) fine-tuning of the convolutional neural network (CNN) to improve the original model trained on the irrelevant labeled dataset. Since the clustering results can be very noisy, we add a selection operation between the clustering and fine-tuning. At the beginning when the model is weak, CNN is fine-tuned on a small amount of reliable examples which locate near to cluster centroids in the feature space. As the model becomes stronger in subsequent iterations, more images are being adaptively selected as CNN training samples. Progressively, pedestrian clustering and the CNN model are improved simultaneously until algorithm convergence. This process is naturally formulated as self-paced learning. We then point out promising directions that may lead to further improvement. Extensive experiments on three large-scale re-ID datasets demonstrate that PUL outputs discriminative features that improve the re-ID accuracy.

I. INTRODUCTION

Person re-identification needs deep representations that transfer to new environments without extensive target-domain labels. PUL addresses this gap by alternating clustering, reliable-sample selection, and CNN fine-tuning, with experiments showing improved cross-dataset accuracy.

  • Deep CNN representations have advanced re-ID, but most methods assume sufficient labeled data from the same environment.
  • PUL transfers a CNN initialized on irrelevant labeled data to an unlabeled target dataset through iterative feature extraction, k-means clustering, sample selection, and fine-tuning.
  • Reliable samples are selected near cluster centroids before fine-tuning, while progressively more data are incorporated as the model strengthens.
  • The paper frames this alternating improvement of clustering and CNN training as self-paced learning that extracts knowledge from noisy clustering results.
  • Experiments on three large-scale datasets show that PUL noticeably improves re-ID accuracy in cross-dataset evaluation.

II. RELATED WORKS

Prior re-ID research spans deep representations, unsupervised handcrafted or dictionary-based methods, and curriculum-inspired learning. The paper positions PUL as an unsupervised deep-learning approach that uses self-paced selection to address noisy latent labels.

  • Deep re-ID methods commonly learn representations with contrastive or softmax losses, enabling efficient retrieval through extracted feature vectors.
  • Earlier unsupervised re-ID methods largely target small datasets and generally do not use deep features.
  • Handcrafted descriptors provide another unsupervised re-ID strategy, using color, texture, segmentation, and related visual cues.
  • Curriculum learning orders examples by known difficulty, whereas self-paced learning estimates sample easiness during training.
  • PUL applies self-paced learning to select reliable samples for CNN fine-tuning and reduce the risk of bad optima or oscillation with latent labels.

III. PROGRESSIVE UNSUPERVISED LEARNING

PUL alternates latent identity clustering, centroid-based reliability selection, and CNN fine-tuning on an unlabeled dataset. Its self-paced mechanism begins with reliable examples and expands training as representations improve.

  • The framework initializes a CNN on an irrelevant labeled dataset, extracts target features, clusters images, selects samples, and repeatedly fine-tunes the model.
  • Because target identities are unlabeled, identity assignments are treated as latent variables in the optimization.
  • The selection indicator v_i determines whether sample x_i is retained as a reliable training example or discarded during fine-tuning.
  • The optimization alternates among clustering labels, sample-selection variables, and CNN classification loss, with cluster centroids defined in feature space.
  • Samples closer to cluster centroids are considered easier and selected under a reliability threshold λ.
  • Each cluster is constrained to contain at least one reliable sample, while the fine-tuning loss excludes samples whose selection indicator is zero.
  • As fine-tuning changes the feature distribution, same-person images move closer to centroids and more challenging samples can be selected.

B. Optimization Procedure

PUL alternates clustering, reliable-sample selection, and CNN optimization until convergence. The selection objective favors training on reliable samples rather than noisy cluster assignments.

  • B. Optimization Procedure: PUL alternates optimization of cluster assignments and centers, selection indicators, and CNN parameters.The clustering subproblem reduces to classic k-means, while CNN optimization uses the clustering and selection results.
  • B. Optimization Procedure: The selection step balances fitting the clustering objective against selecting reliable training samples.The threshold λ controls selection according to each sample’s distance from its corresponding cluster centroid.
  • B. Optimization Procedure: Algorithm 1 initializes the target model from the original model and repeats its procedure while convergence has not been reached.The algorithm takes unlabeled data, a reliability threshold, a cluster count, and the original model as inputs.
  • B. Optimization Procedure: Each cluster receives a center feature, and samples with cosine similarity above λ are selected for fine-tuning.Choosing the nearest feature to each centroid guarantees at least one reliable sample per cluster.
  • B. Optimization Procedure: The optimization jointly improves CNN representations and clustering, making same-person images closer to their cluster centroids.At the beginning of each iteration, the classifier parameters are initialized randomly.

C. Semi-supervised PUL

Semi-supervised PUL incorporates labeled data into the reliable training set at every iteration. It spans supervised learning when all data is labeled and unsupervised clustering when no labels are available.

  • C. Semi-supervised PUL: Labeled data can be added directly to the selected reliable training set during every PUL iteration.This extends the framework from unsupervised to semi-supervised learning.
  • C. Semi-supervised PUL: With all training data labeled, semi-supervised PUL degenerates to supervised learning.No label inference or reliable-sample selection is needed in this case.
  • C. Semi-supervised PUL: With no labeled data, PUL relies on clustering and self-paced learning to obtain reliable training data.The framework therefore covers a continuum from fully supervised to fully unsupervised training.
  • C. Semi-supervised PUL: The working mechanism selects more reliable samples from training iteration 2 to iteration 3 and converges when their number saturates.The figure illustrates this progression for two visual examples, with colors denoting distinct identities and red circles marking reliable areas.

A. Datasets and Settings

The evaluation uses DukeMTMC-reID, Market-1501, and CUHK03, with rank-based accuracy and mAP reported under single-query evaluation. Table II compares cross-dataset PUL with a fine-tuned CNN baseline.

  • A. Datasets and Settings: The study mainly evaluates DukeMTMC-reID and Market-1501 because they are relatively large, multi-camera datasets, and also reports CUHK03.MARS is excluded because its distribution is similar to Market-1501.
  • A. Datasets and Settings: DukeMTMC-reID contains 36,411 images of 1,812 identities captured from 8 viewpoints.It is a subset of the DukeMTMC pedestrian-tracking dataset.
  • A. Datasets and Settings: Market-1501 contains 32,668 images of 1,501 identities captured from 6 cameras.Its images are detected and cropped using the Deformable Part Model.
  • A. Datasets and Settings: CUHK03 contains 14,096 images of 1,467 identities captured from 2 cameras, and experiments use its DPM-detected images.The evaluation follows the train/test protocol proposed in.
  • A. Datasets and Settings: Evaluation reports rank-1, rank-5, rank-10, rank-20 accuracy and mAP for all three datasets using single queries.Table II compares a fine-tuned ResNet-50 baseline with PUL in cross-dataset evaluation.

B. Implementation Details

Experiments use a standardized ResNet-50 training and feature-extraction setup, then compare cross-dataset PUL with a fine-tuned-CNN baseline. PUL improves the reported Market-1501 cross-dataset result over that baseline.

  • B. Implementation Details: The basic CNN is an ImageNet-pretrained ResNet-50 fine-tuned for 40 epochs on each training set.The results are intended as a baseline rather than state-of-the-art performance.
  • B. Implementation Details: Images are resized to 224 × 224, augmented by rotations and shifts, and trained with batch size 16 and momentum 0.9.The learning rate is set to 0.001, and dropout before the fully connected layer is 0.5.
  • B. Implementation Details: Average-pooling outputs provide visual representations, with l2-normalized features used for selection and retrieval but not clustering.Preliminary experiments found l2 normalization inferior for clustering.
  • B. Implementation Details: K-means++ initializes cluster centers, and each clustering step allows up to 300 k-means iterations.This initialization is used to speed convergence.
  • B. Implementation Details: +8.6% in rank-1 accuracy and +5.9% in mAP are reported on Market-1501 when Duke trains the original CNN and PUL is compared with the baseline.The reliability threshold is λ = 0.85, and the cluster count is dataset-dependent.

D. Evaluation of PUL

The evaluation studies reliable-sample selection and the effects of the cluster count K and threshold λ on PUL. Selection is necessary, K = 750 performs best in the tested setting, and λ = 0.85 yields superior accuracy.

  • 1) Ablation study — PUL without sample selection:: Using all clustered images for fine-tuning produces little improvement, showing that reliable sample selection is necessary in PUL.The largest gains over baseline are +1.4% in rank-1 accuracy and +1.2% in mAP when K = 1,250.
  • 2) Further understanding of PUL with parameter changes:: K = 750 achieves the best performance among the tested cluster counts when λ = 0.85 on Market-1501.The tested values are 250, 500, 750, 1,000, and 1,250.
  • 2) Further understanding of PUL with parameter changes:: For λ = 0.85, selected samples increase from 31.8% before the ninth iteration to 45.3%, then oscillate between 45.3% and 47.4%.The selection curve saturates after 20 epochs, suggesting training can stop when the selected-sample count converges.
  • 2) Further understanding of PUL with parameter changes:: When λ ≤ 0.85, the proportion of reliable samples drops sharply after the second iteration because the initially weak model selects too many unreliable samples.After fine-tuning on Market, the model better separates reliable and unreliable samples.
  • 2) Further understanding of PUL with parameter changes:: Performance generally improves over iterations, and λ = 0.85 yields superior accuracy with K = 750.The λ study reports re-ID performance throughout the training process.

E. Semi-supervised Re-ID

The semi-supervised evaluation shows that adding labeled identities improves PUL, while comparisons with non-deep methods favor CNN-based approaches and especially PUL. More labels improve performance but increase labeling cost.

  • E. Semi-supervised Re-ID: Adding 25 or 50 labeled identities to PUL notably improves re-ID accuracy over unsupervised PUL.With 25 labeled IDs, rank-1 gains are +2.2% on Market-1501, +2.7% on Duke, and +0.8% on CUHK03.
  • E. Semi-supervised Re-ID: Using 50 labeled identities improves rank-1 accuracy over using 25 IDs by +3.2% on Market-1501, +3.8% on Duke, and +0.2% on the third testing set.The paper identifies higher labeling cost as the trade-off for using more labeled data.
  • F. Comparison with Non-deep Methods: Initialization with more labeled datasets does not noticeably improve accuracy and can sometimes perform worse than single-dataset initialization.For Duke, PUL reaches 30.0% rank-1 with Market+CUHK03 initialization versus 30.4% with Market alone.
  • F. Comparison with Non-deep Methods: Both the CNN baseline and PUL outperform UMDL by large margins, with PUL showing larger gaps across the three datasets.The CNN baseline gains +5.5%, +4.4%, and +2.9% in rank-1 accuracy, while PUL gains +8.7%, +11.5%, and +6.7%.

V. CONCLUSION

The conclusion presents PUL as an iterative clustering-and-fine-tuning method whose success depends on reliable sample selection. It also identifies video tracklets and camera diversity as directions for extension.

  • V. CONCLUSION: PUL alternates k-means clustering and CNN fine-tuning, with reliable sample selection identified as a key component of its success.The method progressively produces CNN models with high discriminative ability.
  • V. CONCLUSION: The paper suggests using tracklet frames for initialization and incorporating camera diversity into sample selection as possible extensions.These directions target video re-ID and selection of training samples from multiple cameras.
Loading 1705.10444v2…