PaperlayerView exact revision on arXiv

Source-linked AI summary

Person Re-identification with Metric Learning using Privileged Information

Xun Yang, Meng Wang, Dacheng Tao

arXiv:1904.05005v1cs.CV

TL;DR

Person re-identification remains difficult under cross-view appearance variation, and global decision thresholds are limited for complex pairwise relationships. The paper learns locally adaptive metrics using privileged information, extends them to multiple feature views, and reports improvements over global-threshold methods and most state-of-the-art results.

  • Problem

    Person re-identification remains challenging because complex inter-class and intra-class variations limit global-threshold metric learning across camera views.

  • Method

    The method jointly learns original- and privileged-space metrics, using privileged distance as a local training threshold, and extends this scheme to weighted multi-view metric learning.

  • Results

    MVLDML+ achieves a state-of-the-art rank-1 accuracy of 61.4% on CUHK01, improving GOGFusion+XQDA by 3.6%, while broader experiments outperform most state-of-the-art results.

  • Takeaways & Limitations

    Privileged information enables locally adaptive decision rules during training, while multi-view learning exploits complementary feature representations for cross-view re-identification.

Abstract

from arXiv · show

Despite the promising progress made in recent years, person re-identification remains a challenging task due to complex variations in human appearances from different camera views. This paper presents a logistic discriminant metric learning method for this challenging problem. Different with most existing metric learning algorithms, it exploits both original data and auxiliary data during training, which is motivated by the new machine learning paradigm - Learning Using Privileged Information. Such privileged information is a kind of auxiliary knowledge which is only available during training. Our goal is to learn an optimal distance function by constructing a locally adaptive decision rule with the help of privileged information. We jointly learn two distance metrics by minimizing the empirical loss penalizing the difference between the distance in the original space and that in the privileged space. In our setting, the distance in the privileged space functions as a local decision threshold, which guides the decision making in the original space like a teacher. The metric learned from the original space is used to compute the distance between a probe image and a gallery image during testing. In addition, we extend the proposed approach to a multi-view setting which is able to explore the complementation of multiple feature representations. In the multi-view setting, multiple metrics corresponding to different original features are jointly learned, guided by the same privileged information. Besides, an effective iterative optimization scheme is introduced to simultaneously optimize the metrics and the assigned metric weights. Experiment results on several widely-used datasets demonstrate that the proposed approach is superior to global decision threshold based methods and outperforms most state-of-the-art results.

I. INTRODUCTION

Person re-identification remains difficult because appearance changes across camera views, while global-threshold metric learning struggles with complex class variations. The paper introduces privileged-information-guided, locally adaptive metric learning and extends it to multiple feature views.

  • Person re-identification supports security and surveillance but remains challenging because view angle, pose, illumination, and background vary across cameras.
  • Global-threshold pairwise constraints can be suboptimal for real-world re-identification with complex inter-class and intra-class variations.
  • LDML+ jointly exploits original data and training-only privileged information to construct a locally adaptive decision rule.
  • Privileged-space distance replaces the global threshold during training, guiding metric learning in the original space like a teacher.
  • MVLDML+ jointly learns multiple metrics from different original feature spaces under shared privileged guidance to use complementary representations.
  • Experiments report that LDML+ improves global-threshold methods, while MVLDML+ outperforms most state-of-the-art results on several datasets.

B. Person Re-identification

Person re-identification retrieves people across spatially disjoint cameras through feature representation and distance learning. This paper uses privileged information to guide metric learning and jointly learns multiple metrics and their weights for complementary features.

  • Person re-identification retrieves a person of interest across spatially disjoint cameras and can be viewed as an image-retrieval problem.
  • Feature-based methods seek discriminative, invariant representations, but concatenating many low-level descriptors can cause the curse of dimensionality.
  • Distance-learning methods include Mahalanobis-like metrics and discriminative subspaces for recognizing people across disjoint views.
  • The proposed approach incorporates auxiliary knowledge to guide metric learning in the original feature space.
  • Its unified multi-metric scheme simultaneously learns base metrics and metric weights to exploit complementary feature representations.
  • LUPI makes privileged information available only during training, where it can provide additional supervision for learning.
  • The generic metric-learning framework minimizes empirical risk over a PSD Mahalanobis metric using pairwise labels and a decreasing loss function.

IV. THE PROPOSED APPROACH

The proposed LDML+ method uses privileged information during training to replace a global threshold with a locally adaptive decision rule. It jointly learns original- and privileged-space metrics while regularizing the privileged metric so it guides similarity decisions.

  • Each training pair has original and privileged features, and the privileged-space squared distance replaces the global threshold.
  • The privileged-space distance acts as a local threshold that transfers the teacher’s similarity concept to decisions in the original space.
  • The method jointly optimizes positive-semidefinite original and privileged metrics under an empirical objective.
  • The objective combines a log-loss term with privileged-metric regularization, keeping the original metric guided by privileged information.
  • LDML+ exploits auxiliary knowledge to construct a locally adaptive decision rule during training.

B. Multi-view Extension

The multi-view extension, MVLDML+, jointly learns view-specific original metrics guided by one privileged metric. An alternating optimization procedure updates metrics, step sizes, and view weights while preserving positive semidefiniteness.

  • B. Multi-view Extension: MVLDML+ extends LDML+ to multiple original feature representations to exploit their complementary information.
  • B. Multi-view Extension: Training jointly learns multiple original-space metrics and one privileged-space metric.
  • B. Multi-view Extension: The multi-view objective assigns weights to the original-space metrics and uses a^r to ensure every view contributes to the final distance.
  • B. Multi-view Extension: LDML+ is categorized as single-view because only its original feature is used during testing, although two feature types support training.
  • Optimization: Alternating optimization updates the view metrics, privileged metric, and weights in turn.
  • Optimization: Metric updates use dynamically adjusted step sizes, while SVD projection retains positive eigenvectors and yields a low-rank factorization for dimension reduction.

2) Optimization of P:

The privileged metric P is optimized by gradient descent with positive-semidefinite projection while the other variables remain fixed. Its step size uses dynamic search but cannot increase during optimization.

  • 2) Optimization of P:: With the original metrics and weights fixed, P is updated using its objective derivative and gradient descent.
  • 2) Optimization of P:: Each P update applies PSD projection through the optimization procedure.
  • 2) Optimization of P:: The step size for P uses the dynamic search strategy but is not allowed to increase during optimization.
  • 2) Optimization of P:: With the view metrics and P fixed, the weight optimization is transformed into a constrained subproblem.

3) Optimization of a:

The view weights a have a closed-form optimum when the metrics are fixed. The parameter r controls whether weights become equal across views or concentrate on the view with the smallest objective contribution.

  • 3) Optimization of a:: The weight subproblem imposes positive-weight constraints on each view coefficient.
  • 3) Optimization of a:: The derivation obtains a closed-form solution for the view weights.
  • 3) Optimization of a:: Because each F_m is positive, the resulting weights satisfy a_m > 0 and provide the global optimum when the metrics are fixed.
  • 3) Optimization of a:: As r approaches infinity, view weights become equal; as r approaches one, the weight concentrates on the view with minimum J_m.
  • 3) Optimization of a:: LDML+ requires no separate solution because it is a special case of MVLDML+.

D. Person Re-ID

The evaluation uses four benchmark person re-identification datasets with standard protocols, repeated splits for smaller datasets, and CMC-based ranking metrics.

  • Datasets: The evaluation covers VIPeR, CUHK01, PRID450S, and Market-1501 person re-identification datasets.VIPeR, CUHK01, and PRID450S are smaller datasets, while Market-1501 is one of the largest.
  • Datasets: VIPeR contains 632 identities and 1,264 images with viewpoint changes of 90 degrees or more and substantial lighting variation.Its protocol randomly assigns 316 identities to training and 316 to testing.
  • Datasets: Market-1501 contains 32,668 bounding boxes from 1,501 identities across two to six cameras and follows its standard evaluation protocol.The training set has 750 identities, while testing uses 751 identities and one randomly selected query image per camera.
  • Protocols: The three smaller datasets are randomly split into equal training and testing halves, with evaluation repeated 10 times and mean results reported.Single-query matching is used for VIPeR, CUHK01, and PRID450S; Market-1501 reports both single-query and multi-query results.
  • Metrics: CMC evaluates gallery rankings for every probe across all datasets, while mean average precision is also used in the evaluation.Market-1501 additionally reports both single-query and multi-query matching results.

3) Features:

The experiments use GOG descriptors as original features and approximate privileged information with fused visual features. LDML+ improves over global-threshold baselines on the small datasets, while comparisons with XQDA vary by dataset.

  • Features: LDML+ is evaluated with GOGRGB, GOGLab, and GOGHSV as separate original feature representations.These descriptors are extracted from RGB, Lab, and HSV color channels, respectively.
  • Features: Privileged information is approximated by fusing LOMO and FTCNN features for VIPeR, CUHK01, and PRID450S.The paper notes that ideal high-level privileged features such as attributes are difficult to obtain.
  • LDML+ evaluation: LDML+ outperforms MLAPG, LDMLσ=1, and LDML on the three small datasets using the evaluated GOG features.On VIPeR, LDML+ surpasses LDML by 1.71%, 1.3%, and 0.54% at rank-1; on CUHK01, the gains are 3.57%, 3.54%, and 4.09%.
  • LDML+ evaluation: On VIPeR, LDML+ improves over XQDA by 1.42%, 1.55%, and 0.7% at rank-1 for the three evaluated GOG features.On CUHK01, the corresponding improvements are 2.34%, 3.06%, and 4.01%.
  • LDML+ evaluation: XQDA performs better than LDML+ with GOGRGB and GOGLab on PRID450S.The paper attributes this to XQDA's supervised dimensionality reduction, which can retain more discriminative information on small datasets.
  • Baseline analysis: LDML outperforms MLAPG on the three datasets, and its data-dependent global threshold performs better or comparably to LDMLσ=1.The paper links this pattern to LDML's dynamic step-size adaptation and interprets it as evidence favoring data-dependent over data-independent thresholds.
  • Conclusion: The reported results support learning a more reliable metric by using privileged information to construct a locally adaptive decision rule during training.The conclusion is drawn from the Table II comparisons.

2) Evaluation of MVLDML+:

MVLDML+ jointly exploits multiple original feature representations under shared privileged guidance, using weighted distance functions. Across VIPeR, CUHK01, and PRID450S, it achieves competitive or superior rank-1 recognition against fusion baselines and recent state-of-the-art methods.

  • Method: MVLDML+ learns an ensemble of base distance functions from multiple original features, combining them as a weighted sum.Each metric is learned from a single feature while sharing the same privileged information.
  • VIPeR: 48.86% rank-1 accuracy on VIPeR exceeds Ensem-XQDA by 1.14% and Ensem-MLAPG by 1.39%.The result uses the first three GOG descriptors from RGB, Lab, and HSV color spaces.
  • CUHK01 and PRID450S: MVLDML+ improves over Ensem-XQDA and Ensem-MLAPG by over 3% at rank-1 on CUHK01.On PRID450S, the corresponding improvements are 0.27% and 2.04%.
  • Comparison with state-of-the-art results: 50.0% rank-1 recognition on VIPeR is the best reported result in the comparison and exceeds the third-best result by 2.2%.The comparison uses four GOG descriptors; the authors note that dataset splitting differs from one cited result.
  • Comparison with state-of-the-art results: 61.4% rank-1 accuracy on CUHK01 is state of the art and improves GOGFusion+XQDA by 3.6%.On PRID450S, MVLDML+ achieves the second-best rank-1 recognition rate, comparable to a deep-network result.
  • Scope: The reported evaluation uses only four GOG color descriptors, although the method could improve with additional visual features.This bounds the feature coverage of the presented experiments.

C. Experiments on Market-1501

On Market-1501, LDML+ improves single- and multi-query re-identification over global-threshold baselines, while MVLDML+ benefits from complementary feature descriptors and remains competitive with recent results despite using handcrafted features.

  • LDML+ evaluation: LDML+ outperforms baseline methods on Market-1501, including XQDA, across the reported GOG feature settings.In single-query evaluation, LDML+ reaches rank-1 rates of 52.08%, 52.29%, and 45.49%, with corresponding mAP scores of 29.10%, 28.72%, and 23.06%.
  • LDML+ evaluation: 8.79% at rank-1 and 5.84% in mAP separate LDML+ from XQDA with GOGRGB on Market-1501.The authors attribute XQDA’s weaker performance to its Gaussian assumption not holding under the dataset’s complex variations.
  • LDML+ evaluation: LDML+ improves LDML by 4.25%, 3.74%, and 4.58% at rank-1 and by 3.43%, 3.12%, and 2.67% in mAP across the three features.These single-query comparisons support the advantage of the locally adaptive decision rule over the global decision rule.
  • LDML+ evaluation: In multi-query evaluation, LDML+ reaches rank-1 rates of 63.63%, 62.53%, and 57.60% across the three features.Each rate improves on the corresponding single-query result reported for LDML+.
  • MVLDML+ evaluation: MVLDML+ achieves 55.94% rank-1 and 32.19% mAP in single-query evaluation, rising to 66.81% rank-1 and 40.63% mAP in multi-query evaluation.The results indicate that the multi-view formulation exploits complementary information from multiple feature descriptors.
  • Comparison with state-of-the-art results: With re-ranking, MVLDML+ obtains 48.01% and 56.45% mAP and 64.82% and 74.58% rank-1 in single- and multi-query settings.Using handcrafted GOG descriptors, the re-ranked method is reported as competitive with recent state-of-the-art results, including deep models.

D. Analysis of the proposed method

Parameter analyses show that LDML+ is robust to small regularization values, benefits from scaling privileged distances when that information is unreliable, and gains the clearest advantage over baselines at lower PCA dimensions.

  • Effects of β: When privileged information is totally wrong, including scale parameter β significantly improves LDML+ over omitting β.The scale parameter smooths privileged-space distances and helps optimization retain satisfactory performance under severely mismatched privileged information.
  • Effects of λ: For 10^-5 ≤ λ ≤ 10^-2, LDML+ rank-1 performance changes little, whereas performance drops rapidly when λ exceeds 10^-2.Larger λ values accelerate convergence but can cause premature convergence to a poor local optimum.
  • Effects of λ: At λ = 10^-1, the algorithm converges in fewer than 20 iterations but achieves the lowest rank-1 recognition rate.The regularization term therefore presents a performance–convergence trade-off.
  • Performance across PCA dimensions: At PCA dimension 100, LDML+ improves rank-1 performance by over 10% against LDML and 5% against MLAPG.The advantage is smaller at dimension 631, where nearly all PCA energy is retained, and is about 2% at dimension 500.

4) The effects of the privileged metric P:

The privileged metric P is learned jointly with the original-space metric so privileged distances can provide an adapted local threshold rather than being used directly as a Euclidean threshold.

  • Role of P: Metric P supplies the privileged-space distance used as a local decision threshold for guiding the target metric M.The privileged distance is adapted during training rather than treated as a fixed threshold.
  • Effect of learning P: Jointly learning P helps separate positive and negative pairs in the original feature space after metric learning.The corresponding distance histograms show that almost all positive pairs become distinguished from negative pairs on the original space.
  • Effect of learning P: Directly using Euclidean privileged-space distances as thresholds is contrasted with learning P, because original and privileged feature distributions can differ substantially.The comparison evaluates LDML+ with and without learning P using rank-1 recognition and mAP.
  • Role of P: The paper positions the P-to-M transfer as teacher knowledge moving from privileged space to the original space where decisions are made.This training-time guidance is part of the proposed cross-view re-identification approach and its multi-view extension.
Loading 1904.05005v1…