PaperlayerView exact revision on arXiv

Source-linked AI summary

Unsupervised Person Re-identification by Soft Multilabel Learning

Hong-Xing Yu, Wei-Shi Zheng, Ancong Wu, Xiaowei Guo, Shaogang Gong, Jian-Huang Lai

arXiv:1903.06325v2cs.CV

TL;DR

Unsupervised RE-ID lacks pairwise labels and therefore struggles with discriminative learning across disjoint camera views. The paper proposes MAR, which learns soft multilabel representations from auxiliary reference persons and uses them for hard-negative mining and cross-view consistency. MAR outperforms state-of-the-art unsupervised RE-ID methods on Market-1501 and DukeMTMC-reID, while its performance depends on having a sufficiently large reference population.

  • Problem

    Unsupervised RE-ID must learn discriminative information without substantial pairwise labels across camera views, where cross-view appearance variation makes identity discovery difficult.

  • Method

    MAR learns real-valued soft multilabels by comparing unlabeled target persons with auxiliary reference persons, then jointly performs hard-negative mining, cross-view-consistent learning, and reference-agent learning.

  • Results

    MAR outperforms state-of-the-art unsupervised RE-ID methods by significant margins on Market-1501 and DukeMTMC-reID.

  • Takeaways & Limitations

    Soft multilabels provide comparative characteristics that MAR uses to mine target-domain discriminative information beyond direct absolute-feature comparison.

  • Takeaways & Limitations

    MAR performs drastically worse with very few reference persons, such as 100, because the small reference population has poor soft multilabel representation capacity.

Abstract

from arXiv · show

Although unsupervised person re-identification (RE-ID) has drawn increasing research attentions due to its potential to address the scalability problem of supervised RE-ID models, it is very challenging to learn discriminative information in the absence of pairwise labels across disjoint camera views. To overcome this problem, we propose a deep model for the soft multilabel learning for unsupervised RE-ID. The idea is to learn a soft multilabel (real-valued label likelihood vector) for each unlabeled person by comparing (and representing) the unlabeled person with a set of known reference persons from an auxiliary domain. We propose the soft multilabel-guided hard negative mining to learn a discriminative embedding for the unlabeled target domain by exploring the similarity consistency of the visual features and the soft multilabels of unlabeled target pairs. Since most target pairs are cross-view pairs, we develop the cross-view consistent soft multilabel learning to achieve the learning goal that the soft multilabels are consistently good across different camera views. To enable effecient soft multilabel learning, we introduce the reference agent learning to represent each reference person by a reference agent in a joint embedding. We evaluate our unified deep model on Market-1501 and DukeMTMC-reID. Our model outperforms the state-of-the-art unsupervised RE-ID methods by clear margins. Code is available at https://github.com/KovenYu/MAR.

1. Introduction

The paper addresses unsupervised RE-ID without exhaustive pairwise labels by learning soft multilabels from auxiliary reference persons. Its unified MAR model uses these representations for hard-negative mining, cross-view consistency, and discriminative target-domain embeddings.

  • Motivation: Supervised RE-ID requires substantial pairwise labels across camera views, limiting scalability when only unlabeled data are available.Unsupervised methods still struggle to discover identity-discriminative information because of cross-view appearance variation.
  • Soft multilabel learning: Soft multilabel learning represents each unlabeled person with a real-valued label-likelihood vector derived by comparison with known auxiliary reference persons.This mines potential label information without assigning a single pseudo label.
  • Soft multilabel learning: Soft multilabel-guided hard-negative mining uses comparative characteristics to distinguish visually similar but different unlabeled persons.Pairs with similar visual features but dissimilar soft multilabels are treated as probable hard negatives.
  • Unified model: MAR jointly formulates soft multilabel-guided hard-negative mining, cross-view consistent soft multilabel learning, and reference agent learning.Reference agents represent auxiliary reference persons in a joint embedding with unlabeled persons for efficient comparison.
  • Evaluation: Experiments on Market-1501 and DukeMTMC-reID show that MAR outperforms state-of-the-art unsupervised RE-ID methods by significant margins.The contribution passage reports this result across both benchmarks without specifying a single aggregate metric.

2. Related Work

Related unsupervised RE-ID methods cluster unlabeled target data, transfer source knowledge, exploit tracklets, or reduce target labeling. The paper distinguishes MAR from these approaches and from multilabel classification and zero-shot learning.

  • Unsupervised RE-ID: Unsupervised RE-ID methods address unlabeled targets through source-label transfer, target clustering, tracklet associations, or reduced target labeling budgets.These approaches rely on different forms of transferred knowledge, clustering structure, temporal association, or limited annotation.
  • Unsupervised domain adaptation: The paper argues that transferred source knowledge may be less effective after domain shift because prior methods do not mine discriminative information in the unlabeled target domain.The distinction is specifically drawn against unsupervised domain-adaptation approaches.
  • Unsupervised domain adaptation: Standard unsupervised domain adaptation commonly assumes shared classes across source and target domains, unlike RE-ID with completely different source and target persons.This class mismatch limits the direct applicability of those assumptions to the RE-ID setting.
  • Multilabel classification: The paper’s soft multilabel differs from multilabel classification because it is a learned representation by other reference persons, not a ground-truth binary class-membership vector.Consequently, existing multilabel-classification models serve a different purpose.
  • Zero-shot learning: Soft multilabel reference learning resembles zero-shot learning by representing an unknown target person with known reference persons, but lacks predefined semantic attributes.Unlike a related ID-regression approach, it does not require substantial labeled target persons according to the cited comparison.

3. Deep Soft Multilabel Reference Learning

MAR learns soft multilabels for unlabeled target images by comparing them with auxiliary reference agents, then uses these labels to guide discriminative embedding learning. Its unified design combines hard-negative mining, cross-view consistency, and reference-agent learning.

  • 3.1. Problem formulation and Overview: Each target image receives a normalized real-valued label-likelihood vector by comparison with labeled auxiliary reference persons represented as reference agents.The reference population is non-overlapping with the unlabeled target population, and the vector dimensions sum to 1.
  • 3.2. Soft multilabel-guided hard negative mining: Visually similar target pairs are treated as positives when their soft multilabel agreement is high and as hard negatives otherwise.The method compares feature similarity and soft multilabel agreement, selecting the most similar pairs for mining.
  • 3.2. Soft multilabel-guided hard negative mining: The mining procedure selects the top pM feature-similar pairs and assigns them to P only when they also rank among the top pM pairs by soft multilabel agreement.The positive and hard-negative sets are dynamically reconstructed within each training batch using the current embedding.
  • 3.3. Cross-view consistent soft multilabel learning: Cross-view consistent learning aligns soft multilabel distributions across camera views because most RE-ID pairs span different cameras.The loss compares each camera-view distribution with the overall target-domain distribution using a distributional distance.
  • 3.3. Cross-view consistent soft multilabel learning: The cross-view loss uses a simplified 2-Wasserstein distance on log-soft-multilabel mean and standard-deviation vectors, making computation efficient within a batch.The formulation relies on the observed approximate log-normal distribution and uses naturally available camera-view labels.
  • 3.4. Reference agent learning: Reference agents act as compact feature summarizers that should be mutually discriminative while representing all images of their corresponding auxiliary persons.They reside in a shared unit-norm hypersphere embedding with target image features, enabling direct comparison.

4. Experiments

MAR is evaluated on Market-1501 and DukeMTMC-reID without target labels, using auxiliary reference data and comparisons against unsupervised RE-ID baselines. It significantly outperforms prior methods, while ablations and visualizations support its soft multilabel guidance and cross-view design.

  • Evaluation benchmarks: MAR is evaluated on Market-1501 and DukeMTMC-reID, with target identities split between training and testing and no target labels used during training.The evaluation uses Rank-1/Rank-5 matching accuracy and mean average precision (MAP).
  • Comparison to the state of the art: The comparison includes handcrafted-feature, pseudo-label, and unsupervised domain-adaptation methods.Baselines include LOMO, BoW, DIC, ISR, UDML, CAMEL, DECAMEL, PUL, TJ-AIDL, PTGAN, SPGAN, and HHL.
  • Comparison to the state of the art: 20.2%/20.8% on DukeMTMC-reID and 5.5%/8.6% on Market-1501 are MAR’s improvements over HHL in Rank-1 accuracy/MAP.The paper reports these gains as improvements over the current state of the art.
  • Comparison to the state of the art: MAR’s soft multilabel reference learning outperforms pseudo-label methods by mining auxiliary reference information for discriminative clues between visually similar persons.The paper contrasts this with pseudo-label assignment that is blind to such potential discriminative information.
  • Ablation study: 21.5%/24.0% Rank-1 gains over the pretrained model show the effect of soft multilabel-guided hard negative mining on Market-1501/DukeMTMC-reID.The pretrained model has source-domain discriminative training but does not mine discriminative information in the unlabeled target domain.
  • Visual results and insight: The model’s learned soft multilabels identify visually similar references and expose fine-grained clues, such as backpack presence, in hard negative pairs.Low soft multilabel agreement results when similar-looking persons receive high likelihood for different reference persons.
  • Visual results and insight: Performance stabilizes above 1,500 reference persons but drops drastically with only 100, indicating that a very small auxiliary dataset is insufficient.The stable point is approximately twice the number of training persons in both target datasets.

5. Conclusion

MAR uses auxiliary source RE-ID data to mine latent label information in unlabeled target data through unified soft multilabel learning. Experiments on two benchmarks validate MAR and each of its learning components.

  • MAR uses soft multilabels to mine latent discriminative information that direct absolute-feature comparison cannot discover.
  • The unified model simultaneously optimizes soft multilabel-guided hard negative mining, cross-view consistent soft multilabel learning, and reference agent learning.
  • Experiments on two benchmarks validate the effectiveness of MAR and each of its learning components.
Loading 1903.06325v2…