PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Metric Learning for Practical Person Re-Identification

Dong Yi, Zhen Lei, Stan Z. Li

arXiv:1407.4979v1cs.CVcs.LGcs.NE

TL;DR

Person re-identification requires representations and metrics that remain useful across highly variable images and different datasets. The paper proposes a siamese deep metric learning framework that jointly learns image features and similarity, with parameter sharing for general matching. Experiments on VIPeR and PRID report strong intra-dataset and cross-dataset performance, including improvements over existing methods.

  • Problem

    Person re-identification must handle low-resolution, variable images and the under-studied problem of generalizing from a source dataset to a different target dataset.

  • Method

    A siamese convolutional network jointly learns color features, texture features, and a similarity metric, using shared or unshared subnetworks for different matching settings.

  • Results

    The method outperforms or matches state-of-the-art methods in intra-dataset tests and significantly outperforms existing methods in cross-dataset experiments.

  • Takeaways & Limitations

    Deep metric learning provides a unified approach for person re-identification across view and dataset changes within the evaluated VIPeR and PRID settings.

Abstract

from arXiv · show

Various hand-crafted features and metric learning methods prevail in the field of person re-identification. Compared to these methods, this paper proposes a more general way that can learn a similarity metric from image pixels directly. By using a "siamese" deep neural network, the proposed method can jointly learn the color feature, texture feature and metric in a unified framework. The network has a symmetry structure with two sub-networks which are connected by Cosine function. To deal with the big variations of person images, binomial deviance is used to evaluate the cost between similarities and labels, which is proved to be robust to outliers. Compared to existing researches, a more practical setting is studied in the experiments that is training and test on different datasets (cross dataset person re-identification). Both in "intra dataset" and "cross dataset" settings, the superiorities of the proposed method are illustrated on VIPeR and PRID.

I. INTRODUCTION

Person re-identification must handle severe image variation and limited cross-dataset evidence. The paper addresses these challenges with deep metric learning that jointly learns representations and similarity, then evaluates generalization across datasets.

  • Motivation: Surveillance images vary in resolution, illumination, camera view, pose, and non-rigid deformation, making person matching difficult.Images may be around 48 × 128 pixels, with unstable lighting and arbitrary camera directions and poses.
  • Motivation: Cross-dataset re-identification is important because source and target datasets differ in cameras, environments, and probability distributions.The practical setting trains on source-domain datasets and tests on different target-domain datasets.
  • Related work: Existing approaches commonly separate feature extraction from metric learning and combine color and texture features using simple fusion strategies.Hand-crafted features include HSV histograms, Gabor features, and HOG, while metric learning methods include KISSME and RDC.
  • Contribution: Deep metric learning jointly learns color features, texture features, and the similarity metric from image pixels in one framework.Its flexible two-subnetwork structure can support view-specific or general re-identification depending on parameter sharing.
  • Evaluation: The method is evaluated on VIPeR and PRID for intra-dataset recognition and on cross-dataset settings using i-LIDS and CUHK Campus for training.The reported cross-dataset results are significantly better than existing methods under similar settings.

III. DEEP METRIC LEARNING

The proposed deep metric learning system uses paired person images in a siamese convolutional architecture rather than standalone classification. Its subnetworks may share parameters for general matching or remain separate for view-specific tasks.

  • Architecture: Deep learning is used to learn a nonlinear metric suited to resolution, illumination, and pose variation in person images.The method introduces the network architecture, parameters, cost function, and implementation details for deep metric learning.
  • Architecture: A siamese network processes two samples together, mapping an image pair to a similarity label or score.The score supports ranking gallery images by similarity to a probe image.
  • Architecture: The SCNN contains two convolutional subnetworks connected by a connection function that produces the pairwise similarity.The figure describes the broader structure as CNN components, a connection function, and a cost function.
  • Architecture: Parameter sharing determines whether the network operates in a view-specific mode or a general mode.Without sharing, the network is suited to view-specific matching; with sharing, it is more appropriate for general tasks such as cross-dataset re-identification.

B. Convolutional Neural Network

The proposed convolutional network uses a siamese, part-based architecture with learned similarity and cost functions. Cosine similarity and binomial deviance are selected to compare samples and emphasize difficult pairs during training.

  • Architecture: The CNN uses two convolutional layers, two max pooling layers, and a fully connected layer, producing a 500-dimensional vector.Both convolutional and pooling layers use 64 channels; filter sizes are 7 × 7 and 5 × 5.
  • Architecture: Three overlapped body parts share the first convolutional layer, learn part-specific filters, and fuse high-level features by sum rule.Joint part training reduces complexity and enables feature-level fusion.
  • Siamese design: The siamese structure comprises two subnetworks, a connection function, and a cost function that converts sample relationships into training costs.The connection function evaluates relationships between samples, while the cost function guides optimization.
  • Similarity function: Cosine is chosen as the connection function because it is bounded to [-1, 1] and invariant to sample magnitude.These properties are presented as advantages over unbounded Euclidean distance and nondifferentiable absolute difference.
  • Cost function: Binomial deviance is compared with Fisher criterion, with the former focusing training on false-classified or boundary-near samples.Its parameters include α and β, while the objective separates positive and negative pairs.
  • Learning: Matrix-form cost and gradient computation lets a batch of 128 generate 128 × (128 −1)/2 = 8128 sample pairs, accelerating data scanning and saving training time.This replaces the earlier pairwise formulation and supports more pair processing per mini-batch.

IV. EXPERIMENTS

Experiments evaluate the method on VIPeR and PRID under intra-dataset and cross-dataset settings. The cross-dataset protocols train on i-LIDS or CUHK Campus and test on VIPeR or PRID to assess generalization ability.

  • Evaluation settings: Experiments use VIPeR and PRID because their single-shot evaluation protocols are described as the clearest among the considered datasets.The study includes both intra-dataset and cross-dataset settings.
  • Intra dataset: Intra-dataset experiments train and test on VIPeR or train and test on PRID.These experiments illustrate the basic performance of the proposed method.
  • Cross dataset: Cross-dataset experiments train on i-LIDS or CUHK Campus and test on VIPeR or PRID.These experiments are intended to illustrate generalization ability.

A. Intra Dataset

The intra-dataset protocols split VIPeR and PRID into training and testing subjects, repeat the process 11 times, and reserve the first split for parameter tuning. Training merges images from both camera views and labels pairs by subject identity.

  • VIPeR protocol: VIPeR contains 632 subjects with two images per subject from two cameras, split into 316 training and 316 testing subjects across 11 random repetitions.The first split is used for parameter tuning.
  • PRID protocol: PRID contains 385 subjects in camera A and 749 in camera B, with the first 200 subjects appearing in both cameras.The protocol randomly selects 100 shared subjects for training and uses the remaining subjects for testing.
  • Training procedure: Training images from both cameras are merged and shuffled, while same-subject pairs receive 1 and different-subject pairs receive -1.The symmetric mask matrix sets its lower triangular entries to 0 to avoid redundant computation.
  • Development evaluation: Before test-view evaluation, the study examines data augmentation, asymmetric pair costs, and cost-function choice on the development view.The cost-function parameters are set to α = 2 and β = 0.5.

1) Data Augmentation:

The study mirrors person images to enlarge both training and test sets, then fuses the resulting similarity scores. Results report significant gains, especially at top ranks, indicating the importance of dataset scale for training.

  • Data augmentation: Mirroring increases each subject’s training images from 2 to 4 and produces four test similarity scores from two originals and their mirrored versions.The final test score is fused by sum rule.
  • Results: Significant improvements from data augmentation occur especially at the top ranks.The passage links this result to the importance of dataset scale for training good networks.

2) Asymmetric Cost:

The experiments address negative-pair under-fitting by weighting negative pairs more heavily and compare Binomial deviance with Fisher criterion on VIPeR.

  • Batch construction leaves many negative pairs unused, making negative-pair under-fitting a concern.
  • The method balances pair types by fixing positive-pair cost at 1 and tuning the negative-pair cost c from 1 to 3.
  • At c = 2, Dev. view of VIPeR achieves the highest overall recognition performance.
  • Binomial deviance produces a smaller training–test cost gap, whereas Fisher criterion shows a larger gap indicating easier overfitting to training data.
  • Binomial deviance is more suitable for person re-identification because it emphasizes boundary samples and is less affected by positive- and negative-sample distributions.

4) Results:

The evaluation uses tuned Improved DML configurations in intra- and cross-dataset experiments, including VIPeR, PRID, i-LIDS, and CUHK Campus. Results report strong within-dataset performance and dataset-dependent cross-dataset behavior.

  • The final configuration mirrors images, sets c = 2 and epoch = 180, and uses Cosine with Binomial Deviance; results average 10 repetitions.
  • Improved DML increases VIPeR rank-1 recognition from 28.23% to 34.4% while achieving 5× training speed over the pairwise version.
  • On PRID, the proposed method outperforms RPML, but its advantage is smaller at 1% to 3% than the 4% to 10% advantage on VIPeR.
  • Cross-dataset experiments train on i-LIDS or CUHK Campus and test on VIPeR or PRID without using target-domain samples for classifier adaptation.
  • The cross-dataset setup spans different environments and devices, with i-LIDS containing 119 people and CUHK Campus 1816 subjects.

1) Results:

The method is evaluated in intra-dataset and cross-dataset person re-identification experiments on VIPeR and PRID. Cross-dataset performance declines sharply, but combining networks can improve recognition in some transfer settings.

  • Cross-dataset results: When trained on i-LIDS or CUHK Campus and tested on VIPeR, the method slightly outperforms original DML, matches DTRSVM in one setting, and improves further when similarity scores are fused.CUHK Campus training produces a significant improvement, while score fusion approaches some intra-dataset methods.
  • Cross-dataset results: When tested on PRID, i-LIDS training significantly outperforms DTRSVM, whereas CUHK Campus training decreases recognition rates markedly.The decline is attributed to the large difference between CUHK Campus and PRID.
  • Cross-dataset results: Fusing i-LIDS and CUHK Campus similarity scores on PRID raises rank-1 recognition from 8% to 13.8%.The paper reports that score fusion increases performance, especially for rank-1 recognition.
  • Generalization limitation: Cross-dataset rank-30 recognition rates on VIPeR and PRID both drop by more than 20% relative to intra-dataset experiments.The results indicate that models trained on one dataset generalize poorly across datasets with distinctive properties.
  • Evaluation scope: The proposed siamese convolutional network is evaluated in both intra-dataset and cross-dataset person re-identification experiments.The experiments are conducted on VIPeR and PRID, with cross-dataset training and testing also examined.
  • Conclusion: The conclusion reports that the network addresses cross-view and cross-dataset person re-identification and outperforms state-of-the-art methods significantly.The paper identifies generalization across views and datasets as an ongoing research direction.

APPENDIX A GRADIENTS OF GENERAL SCNN

Appendix A derives gradients for the general SCNN, whose parameter-sharing structure gives the two sub-networks the same output for each sample. The derivation expands cosine similarity and propagates derivatives through the cost function.

  • General SCNN setup: In the general SCNN, the two sub-networks share parameters and therefore produce the same output for a sample.The training-set outputs are denoted X, and the cost is defined over the training set.
  • Gradient terms: A_ij is treated as a weight for each sample pair in the gradient computation.The derivative of similarity S_ij with respect to X is also used in the derivation.
  • Cosine-similarity derivatives: The appendix expands cosine similarity S_ij and derives the associated derivatives of x_j^T x_j with respect to X.B_ij, C_ij, and D_ij are defined in earlier equations and substituted into the subsequent gradient expressions.
  • Final gradient: Substituting the intermediate derivatives into the preceding equations yields the final formulation for the gradient of J_dev with respect to X.The final expression uses g_AC and d_AD as defined in earlier equations.

APPENDIX B GRADIENTS OF VIEW SPECIFIC SCNN

Appendix B gives gradient expressions for a view-specific SCNN with separate outputs for two views. The corresponding similarity and mask matrices are rectangular, reflecting the differing sample counts between views.

  • Gradient dimensions: The similarity matrix S and mask matrix M have dimensions n × m, where n and m are the sample counts in the two views.The appendix derives gradients of J_dev with respect to both X and Y.
Loading 1407.4979v1…