PaperlayerView exact revision on arXiv

Source-linked AI summary

Unsupervised Person Re-identification by Deep Asymmetric Metric Embedding

Hong-Xing Yu, Ancong Wu, Wei-Shi Zheng

arXiv:1901.10177v1cs.CV

TL;DR

Unsupervised Re-ID is constrained by costly labels and view-specific distortions that hinder universal representations. The paper introduces DECAMEL, which jointly learns features and an asymmetric metric through cross-view clustering. Experiments on seven benchmark datasets show effectiveness and noticeable-margin improvements over state-of-the-art methods.

  • Problem

    Expensive cross-view labels limit supervised Re-ID, while view-specific conditions make universal projections inadequate for extracting cross-view discriminative information from unlabelled data.

  • Method

    DECAMEL jointly learns feature representations and view-specific asymmetric metric transformations through an unsupervised deep framework based on cross-view clustering.

  • Results

    Experiments on seven size-varying benchmark datasets show noticeable-margin improvements over state-of-the-art unsupervised Re-ID models.

  • Takeaways & Limitations

    Asymmetric modeling provides a framework for learning compact cross-view cluster structure and mining underlying cross-view discriminative information in unsupervised Re-ID.

  • Takeaways & Limitations

    The training and testing camera views are assumed to be the same; view-imbalance robustness lacks theoretical guarantees.

Abstract

from arXiv · show

Person re-identification (Re-ID) aims to match identities across non-overlapping camera views. Researchers have proposed many supervised Re-ID models which require quantities of cross-view pairwise labelled data. This limits their scalabilities to many applications where a large amount of data from multiple disjoint camera views is available but unlabelled. Although some unsupervised Re-ID models have been proposed to address the scalability problem, they often suffer from the view-specific bias problem which is caused by dramatic variances across different camera views, e.g., different illumination, viewpoints and occlusion. The dramatic variances induce specific feature distortions in different camera views, which can be very disturbing in finding cross-view discriminative information for Re-ID in the unsupervised scenarios, since no label information is available to help alleviate the bias. We propose to explicitly address this problem by learning an unsupervised asymmetric distance metric based on cross-view clustering. The asymmetric distance metric allows specific feature transformations for each camera view to tackle the specific feature distortions. We then design a novel unsupervised loss function to embed the asymmetric metric into a deep neural network, and therefore develop a novel unsupervised deep framework named the DEep Clustering-based Asymmetric MEtric Learning (DECAMEL). In such a way, DECAMEL jointly learns the feature representation and the unsupervised asymmetric metric. DECAMEL learns a compact cross-view cluster structure of Re-ID data, and thus help alleviate the view-specific bias and facilitate mining the potential cross-view discriminative information for unsupervised Re-ID. Extensive experiments on seven benchmark datasets whose sizes span several orders show the effectiveness of our framework.

1 INTRODUCTION

Unsupervised Re-ID is needed because labeled cross-view data is costly, but view-specific distortions make universal representations inadequate. DECAMEL addresses this by jointly learning view-specific asymmetric metrics, features, and cross-view cluster structure.

  • Motivation: Person re-identification matches pedestrians across multiple non-overlapping camera views, but supervised models require expensive cross-view labels.Large quantities of available surveillance data remain unlabelled, limiting supervised methods.
  • Motivation: View-specific illumination, viewpoints, and feature distortions make universal projections struggle to capture cross-view discriminative information without labels.These distortions can make correct cross-view matches rank poorly.
  • Approach: The paper formulates unsupervised Re-ID as asymmetric metric learning, using view-specific transformations to address camera-dependent feature distortions.Each camera view has a corresponding transformation, rather than sharing one universal transformation.
  • Approach: CAMEL learns an asymmetric metric and cluster separation, while DECAMEL jointly learns the feature representation and asymmetric metric end to end.DECAMEL embeds the metric into a deep network through joint learning.
  • Contribution: DECAMEL learns a compact cross-view cluster structure intended to mine underlying cross-view discriminative information for matching.The framework combines feature learning, metric learning, and cluster separation in unsupervised Re-ID.
  • Evaluation: For large-scale view-extendable scenarios, View Clustering provides control over generalizability versus precise modeling of view-specific conditions, and experiments span seven size-varying datasets.The reported experiments show noticeable-margin improvements over state-of-the-art unsupervised Re-ID models.

2 RELATED WORK

Prior unsupervised Re-ID methods often learn universal projections or standard symmetric metrics, whereas DECAMEL models view-specific bias with an asymmetric metric and cross-view clustering. It further embeds this metric into deep networks for joint feature and metric learning.

  • Unsupervised Re-ID models: Existing unsupervised Re-ID work includes feature-representation learning and dictionary-learning approaches.These categories focus on discriminative or invariant features, or dictionaries whose atoms represent semantic elements.
  • Unsupervised Re-ID models: DECAMEL learns view-specific transformations to address view-specific bias while preserving cross-view consistency through asymmetric metric learning.Unlike RKSL, it discovers cross-view cluster structure rather than only maximizing correlations between separately learned projections.
  • Unsupervised Re-ID models: Compared with CAMEL, DECAMEL jointly learns feature representation and metric in an unsupervised deep framework.The framework uses a novel loss function, architecture, and learning algorithm, and adds view clustering to improve generalizability and scalability.
  • Unsupervised metric learning: Unlike AML, DECAMEL performs cross-view clustering, explicitly models view-specific bias, and embeds asymmetric metric learning into a deep neural network.AML projects data into a low-dimensional space and minimizes within-cluster squared error using a Mahalanobis distance.
  • Unsupervised deep clustering embedding: DECAMEL combines unsupervised asymmetric metric learning with cross-view clustering, whereas related deep clustering models use standard symmetric metrics.Its framework first learns an initial asymmetric metric with CAMEL and then jointly learns feature and metric in a deep network.

3 APPROACH

The approach first learns an unsupervised asymmetric metric that uses view-specific transformations and cross-view consistency, then embeds it into a deep network through joint feature–metric learning. DECAMEL extends this framework to clustered or unseen views while balancing view-specific modeling and generalizability.

  • Framework: DECAMEL first learns an initial asymmetric metric with CAMEL, then jointly learns the feature representation and metric in a deep network.The framework is based on asymmetric metric clustering and embeds the metric into the network through joint learning.
  • Asymmetric metric learning: View-specific transformations address camera-dependent distortions that can dominate clustering across lighting, pose, and occlusion changes.The method replaces a universal transformation with transformations specific to each camera view.
  • Asymmetric metric learning: Cross-view consistency regularization limits discrepancies between view-specific transformations, making the asymmetric metric more principled.The regularizer controls the Frobenius-norm discrepancy between transformations and its scaled upper bound on coincidence discrepancy.
  • Deep embedding: DECAMEL jointly learns nonlinear features and the asymmetric metric, overcoming CAMEL’s linear and separate feature-extraction limitations.This joint learning is intended to produce a better cross-view cluster structure.
  • View clustering: View clustering assigns unseen views to the most similar learned cluster and reuses that cluster’s feature transformation.With J view clusters, the method can generalize transformations to unseen views; setting J=V recovers DECAMEL, while J=1 yields a universal transformation.

4 INSIGHT UNDERSTANDING

The visual analyses show that asymmetric transformations reduce view-specific bias more effectively than a symmetric transformation and that DECAMEL progressively improves cross-view clustering through joint learning.

  • Metric initialization: The original feature space shows severe view-specific bias, while a symmetric shared-space transformation does not alleviate it.The two camera views remain roughly separable after the learned view-generic transformation.
  • Metric initialization: The asymmetric shared space produces substantially more overlap between data points from the two camera views.This visual comparison indicates that the view-specific transformations alleviate the bias seen in the original space.
  • Progressive learning: CAMEL provides an initial cross-view cluster structure by bringing same-identity points from different views roughly closer together.The initialization comparison contrasts the original feature space with the space after the initial metric layer.
  • Progressive learning: DECAMEL progressively learns a better cross-view cluster structure by jointly updating the feature space and asymmetric metric across stages.Figure 4 presents synchronized feature-space and shared-space distributions from initialization through convergence.

5 EXPERIMENTS

The experiments compare DECAMEL with related unsupervised models and provide further validation and analysis of the framework.

  • Evaluation: The experiments evaluate DECAMEL against related unsupervised models to assess its effectiveness.The section also reports experimental validations and analyses for understanding the framework.

5.1 Datasets

Experiments use seven Re-ID datasets spanning hundreds to hundreds of thousands of images and identities, with varied cross-view conditions. The datasets include both widely used small benchmarks and larger-scale collections.

  • Seven datasets span scales from hundreds to hundreds of thousands, supporting evaluation of unsupervised Re-ID across problem sizes.VIPeR is small but widely used, while larger datasets are emphasized because unsupervised methods are more meaningful at scale.
  • VIPeR contains 1,264 images, with two images per identity captured from two camera views.
  • CUHK01 contains 3,884 images from 971 identities across two disjoint views, while CUHK03 contains 13,164 images from 1,360 pedestrians across six views.
  • SYSU includes 24,448 RGB images of 502 persons, contrasting frontal or back views with mostly side views across two cameras.
  • Market contains 32,668 images of 1,501 pedestrians from up to six cameras, and includes badly detected distractor samples.
  • ExMarket combines Market with MARS data, while MSMT17 contains 126,441 images of 4,101 persons from 15 views and four days.MSMT17 exhibits extreme lighting variations across camera views.

5.2 Settings

The evaluation follows dataset-specific protocols covering single-shot and multi-shot settings, with rank-1 matching rate and MAP used where required. Training and testing splits are repeatedly sampled or follow recommended protocols.

  • VIPeR uses repeated random half-splits, with one half for training and one for testing, and permits only single-shot evaluation.The procedure is repeated 10 times to obtain average performance.
  • CUHK01 randomly assigns 485 identities to training and 486 to testing, repeating evaluation 10 times in both single-shot and multi-shot settings.
  • CUHK03 follows its recommended protocol, training on 1,260 persons and testing on the remainder, with 10 repetitions and both shot settings.
  • SYSU splits pedestrians equally for training and testing, then repeatedly evaluates single-shot galleries of one image and multi-shot galleries of three images.Testing is repeated 10 times.
  • Market and ExMarket use multi-shot evaluation with CMC and MAP because each identity has images from up to six views.Their protocols are identical because the datasets contain the same identities.
  • Comparisons report single-shot and multi-shot rank-1 matching rates and MAP, alongside CMC curves for single-shot CUHK01, CUHK03, and SYSU.The implementation uses fixed λ = 0.01, K = 500, 10,000 SGD iterations, and batch size 216.

5.3 Comparison to Related Unsupervised Models

DECAMEL is compared with unsupervised Re-ID, published state-of-the-art, and clustering-based metric models across benchmark datasets. It outperforms the compared methods, with especially notable gains where cross-view variation is severe.

  • Related unsupervised models: DECAMEL outperforms related unsupervised models on all datasets in both single-shot and multi-shot settings.Its CMC curves also exceed competing models by large margins at every rank.
  • Related unsupervised models: DECAMEL’s gains are especially significant on CUHK01 and SYSU, where cross-view condition variations are particularly severe.The framework learns asymmetric metrics and compact cross-view clusters, whereas the compared alternatives do not fully address view-specific bias or cluster compactness.
  • Published state-of-the-art: DECAMEL outperforms published unsupervised state-of-the-art methods on VIPeR, CUHK01, and CUHK03 in the single-shot setting.The comparison is limited to these datasets because the cited methods lack reported results on SYSU, Market, and ExMarket.
  • Clustering-based metric models: DECAMEL achieves notable improvements over AML and UsNCA, two clustering-based metric models.The comparison attributes the advantage to accounting for view-specific bias when learning the cross-view cluster structure.

5.4 Further Analysis of DECAMEL

The analysis evaluates DECAMEL’s asymmetric modelling, clustering, component contributions, initialization, and ability to exploit additional labelled or unlabelled data. Results show that moderate regularization and cluster counts are preferable, while joint learning and CAMEL initialization improve performance.

  • Asymmetric vs. Symmetric Modelling: DECAMEL achieves much higher performance than its symmetric counterpart DECMEL, supporting the value of asymmetric modelling.
  • Parameter Evaluation: Moderate λ values yield relatively stable matching rates, whereas excessively large or small λ values reduce performance.Large λ makes the model more symmetric; λ = 0 causes learning failure and collapsed cross-view distributions.
  • Parameter Evaluation: Matching rates remain tightly clustered for K = 300–700, but the extremes K = 1 and K = 1940 cause performance drops.K = 1 collapses all data points toward a single centroid.
  • Parameter Evaluation: The number of clusters containing multiple persons decreases from initialization to convergence, helping explain mild robustness to K.
  • Component-wise Evaluation: Jointly learning feature representation and metric outperforms learning either component alone, indicating that both components contribute to DECAMEL.
  • Metric Initialization: DECAMEL outperforms identity and Xavier metric initialization because CAMEL captures cross-view appearance variations for unsupervised feature learning.
  • Extra Labelled Data: Adding 10%, 20%, or 30% labelled training samples improves both accuracy and MAP on Market.
  • More Unlabelled Data: Increasing the training-set size on ExMarket significantly improves accuracy and MAP.ExMarket provides over 100,000 samples.

5.5 Evaluation on the View Clustering

The view-clustering extension is evaluated when test-time camera views are unseen during training. DECAMEL with view clustering outperforms comparison methods, and its performance approaches the conventional setting as unseen-view generalization improves.

  • View-extendable Re-ID: DECAMEL with View Clustering outperforms the compared methods in the view-extendable setting.
  • View-extendable Re-ID: Performance is lower than in the conventional setting because training samples from the testing views are unavailable.
  • Generalizability of View Clustering: Performance increases when more training views are available because view prototypes cover a wider range of view-specific conditions.
  • Generalizability of View Clustering: DECAMEL with View Clustering performs close to DECAMEL even when testing views are unseen during training.

6 CONCLUSION AND DISCUSSION

The paper formulates unsupervised Re-ID as asymmetric metric learning and develops DECAMEL to jointly learn the metric and deep feature representation. Experiments support its effectiveness, while robustness to heavily imbalanced camera-view data remains a future direction.

  • DECAMEL jointly learns an asymmetric metric and embeds it into a deep feature-learning network through end-to-end unsupervised learning.
  • Experiments show that DECAMEL outperforms related unsupervised Re-ID models.
  • The paper identifies heavily imbalanced camera-view sample counts as a future setting requiring further study and theoretical robustness guarantees.
Loading 1901.10177v1…