PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-pseudo Regularized Label for Generated Data in Person Re-Identification

Yan Huang, Jinsong Xu, Qiang Wu, Zhedong Zheng, Zhaoxiang Zhang, Jian Zhang

arXiv:1801.06742v3cs.CV

TL;DR

Person re-ID models need labeled images, but collecting identity labels across camera views is expensive and leaves training data limited. The paper assigns GAN-generated data MpRL virtual labels reflecting their affiliations with real-data classes, then evaluates the approach with two CNNs. The authors report improved re-ID performance and better improvements than LSRO.

  • Problem

    Expensive manual labeling limits the available images for training person re-ID models, while generated data usually lacks annotation labels.

    Source: §I · p. 1

  • Method

    MpRL assigns each GAN-generated sample class-affiliation weights and uses the samples alongside real data in semi-supervised CNN training.

    Source: §I · pp. 1–2

  • Results

    MpRL improves re-ID performance for two CNNs, and the authors report better improvements than LSRO.

    Source: §VI · p. 12

  • Takeaways & Limitations

    The authors conclude that generated data assigned MpRL labels can improve person re-ID models trained with limited labeled real data.

    Source: §I · p. 1, §VI · p. 12

Abstract

from arXiv · show

Sufficient training data normally is required to train deeply learned models. However, due to the expensive manual process for labelling large number of images, the amount of available training data is always limited. To produce more data for training a deep network, Generative Adversarial Network (GAN) can be used to generate artificial sample data. However, the generated data usually does not have annotation labels. To solve this problem, in this paper, we propose a virtual label called Multi-pseudo Regularized Label (MpRL) and assign it to the generated data. With MpRL, the generated data will be used as the supplementary of real training data to train a deep neural network in a semi-supervised learning fashion. To build the corresponding relationship between the real data and generated data, MpRL assigns each generated data a proper virtual label which reflects the likelihood of the affiliation of the generated data to pre-defined training classes in the real data domain. Unlike the traditional label which usually is a single integral number, the virtual label proposed in this work is a set of weight-based values each individual of which is a number in (0,1] called multi-pseudo label and reflects the degree of relation between each generated data to every pre-defined class of real data. A comprehensive evaluation is carried out by adopting two state-of-the-art convolutional neural networks (CNNs) in our experiments to verify the effectiveness of MpRL. Experiments demonstrate that by assigning MpRL to generated data, we can further improve the person re-ID performance on five re-ID datasets, i.e., Market-1501, DukeMTMC-reID, CUHK03, VIPeR, and CUHK01. The proposed method obtains +6.29%, +6.30%, +5.58%, +5.84%, and +3.48% improvements in rank-1 accuracy over a strong CNN baseline on the five datasets respectively, and outperforms state-of-the-art methods.

I. INTRODUCTION

Person re-identification needs labeled images across camera views, but manual identity labeling leaves relatively few images per identity. This section motivates MpRL as a way to use generated data with labels that reflect differences among generated images and their relationships to real-data classes.

I. INTRODUCTION

  • Manual cross-camera labeling is expensive, leaving limited images per identity even in large re-ID datasets.The paper identifies sufficient labeled training data as essential for deep models.Source: §I · p. 1
  • Generated data could supplement scarce real data, but the paper focuses on how to use GAN-generated samples in semi-supervised training.Generated samples participate directly alongside limited labeled real data.Source: §I · p. 1
  • LSRO assigns every generated sample the same uniform distribution, while MpRL assigns class-affiliation weights that vary by sample.The motivation is that GAN training uses random real-data mini-batches, and visually different generated images may warrant different labels.Source: §I · pp. 1–2
  • MpRL is intended to distinguish generated data and mitigate ambiguous predictions during training.The paper also describes qualitative and quantitative evaluations across large- and small-scale datasets, CNN models, and two GAN-generated data groups.Source: §I · p. 2

A. Semi-supervised Learning

The review situates MpRL among semi-supervised methods that combine labeled and unlabeled data and among person re-ID approaches using deep models. It frames the paper's focus as using generated data and MpRL to boost off-the-shelf CNNs.

A. Semi-supervised Learning

  • Semi-supervised learning combines labeled and unlabeled data, and prior work applies it across segmentation, person identification, multi-label classification, and text classification.The review notes that such methods may be less useful when real data is scarce or difficult to obtain; this paper instead generates unlabeled data with GANs.Source: A. Semi-supervised Learning · p. 2
  • Prior virtual-label strategies include treating generated samples as an extra class, assigning a dynamically updated one-hot pseudo-label, or using LSRO's uniform distribution.The paper notes that one-hot pseudo-labeling may lead to over-fitting and identifies LSRO as the strongest of these comparison methods for boosting re-ID performance.Source: A. Semi-supervised Learning · p. 3

B. Person Re-identification

  • Person re-ID literature includes traditional feature and metric learning, deep identification and verification models, and boosting methods such as re-ranking.This work adopts Identif and Two-stream CNNs to test MpRL and aims to boost off-the-shelf CNN performance with generated data.Source: B. Person Re-identification · p. 3

III. THE PROPOSED MULTI-PSEUDO REGULARIZED LABEL

The method section revisits LSRO before introducing MpRL and its training strategies. LSRO assigns generated images a uniform label distribution across predefined real-data classes.

III. THE PROPOSED MULTI-PSEUDO REGULARIZED LABEL

  • The section revisits LSRO, introduces MpRL, and then presents three training strategies for MpRL.Source: §III · p. 3

A. LSRO for Person Re-ID Revisit

  • LSRO treats generated data as belonging to none of the predefined classes and assigns each class an equal label weight.The paper relates this uniform distribution to label smoothing regularization, which lowers confidence in the ground-truth label and assigns small weights to other classes.Source: A. LSRO for Person Re-ID Revisit · pp. 3–4
  • For an LSRO generated image, the method defines a label distribution over the K predefined real-data classes.Here K is the number of predefined training classes, and k indexes one such class.Source: A. LSRO for Person Re-ID Revisit · p. 4

B. Multi-pseudo Regularized Label

MpRL assigns generated images virtual labels made of different weights over the pre-defined training classes. It derives those weights from predicted class probabilities and regularizes them so contributions from all classes are retained.

B. Multi-pseudo Regularized Label

  • MpRL assigns each generated image a multiple-distributed virtual label using different weights for all pre-defined training classes.The weights are recorded in dictionary α and reflect the generated image’s relation to those classes.Source: B. Multi-pseudo Regularized Label · p. 4
  • Predicted probabilities over the K classes are sorted from minimum to maximum, and their corresponding indices are used to obtain α_k.The resulting weights build the correspondence between generated images and real-data classes.Source: B. Multi-pseudo Regularized Label · p. 4
  • MpRL uses all predicted probabilities rather than only the maximum, and regularizes gaps between contiguous weights to 1/K to retain contributions from all classes.This is intended to address over-fitting when some class weights grow while others become small.Source: B. Multi-pseudo Regularized Label · p. 4
  • For real images, the loss is equivalent to softmax loss; for generated images, only MpRL is used, with separate losses for real and generated data.The loss uses a normalization factor σ and a trade-off parameter λ, set to 1 unless specified otherwise.Source: B. Multi-pseudo Regularized Label · p. 4

C. Training Strategy

The section compares one static MpRL strategy with two dynamic strategies that update generated-image labels during training. The dynamic variants differ in when updating begins.

C. Training Strategy

  • The section evaluates three MpRL training strategies: one static approach and two dynamic approaches with iterative updates.The dynamic approaches differ in whether updates begin from scratch or after an intermediate point.Source: C. Training Strategy · p. 5
  • Static MpRL assigns labels before training using a pre-trained Identif network, then keeps them unchanged throughout training.Predicted probabilities are calculated for each generated image and used to assign its label; this resembles LSRO but uses different class weights.Source: C. Training Strategy · p. 5
  • Dynamic MpRL-I updates labels iteratively from the first training mini-batch, so the same generated image may receive different labels over training.Generated data receive random dMpRL-I labels in the first iteration.Source: C. Training Strategy · p. 5
  • Dynamic MpRL-II assigns dMpRL-IIs to generated data after 20 epochs, when the CNN is relatively stable, updates them iteratively, and sets generated/real-data losses to 0.1/1, so λ = 0.1.Source: C. Training Strategy · p. 5

IV. WHY MULTI-PSEUDO REGULARIZED LABEL WORKS BETTER?

The section contrasts MpRL with all-in-one, one-hot pseudo, and LSRO labels along label distribution, label assignment, and class weighting. It also outlines the training loop for the dynamic MpRL-II strategy.

IV. WHY MULTI-PSEUDO REGULARIZED LABEL WORKS BETTER?

  • The dMpRL-II training loop merges real and generated data, calculates separate losses for each, and assigns MpRL to generated data.The algorithm shuffles the merged dataset and selects real and generated data during its training iterations.Source: §IV · p. 5
  • All-in-one assigns generated images a new class, one-hot pseudo uses the maximum-probability class, LSRO uses a uniform 1/K distribution, and MpRL uses different weights across classes.The section frames these methods as comparisons of label distributions and class weights.Source: §IV · p. 5
  • All-in-one and LSRO assign the same virtual label across generated data, whereas one-hot pseudo and MpRL assign different labels to different generated images.The section identifies same-versus-different virtual labels as one of its three qualitative comparison aspects.Source: §IV · p. 5
  • The section compares one-hot and multiple-label distributions, same and different virtual labels, and equal and different weights across pre-defined classes.These are presented as qualitative discussions, with numerical evidence deferred to the experiments.Source: §IV · p. 5

A. One-hot vs. Multiple Label Distribution

One-hot pseudo labels retain only the highest-probability class, while multiple-label distributions use several class weights. MpRL follows the multiple-label approach.

A. One-hot vs. Multiple Label Distribution

  • Multiple-label distributions retain information from all pre-defined classes, whereas one-hot pseudo labels use only the maximum-probability class.The paper says the multiple-label distribution discourages tuning toward one particular class and reduces the chance of over-fitting.Source: A. One-hot vs. Multiple Label… · pp. 5–6
  • MpRL’s generated-image label assigns different weights across all pre-defined training classes, unlike one-hot pseudo’s single activated class.Figure 2 contrasts the one-hot pseudo and MpRL label distributions.Source: Figure 2 · p. 6
  • The paper reports experiments demonstrating the superiority of using a multiple-label distribution.Source: A. One-hot vs. Multiple Label… · p. 6

B. The Same vs. Different Virtual Labels

MpRL assigns generated images different class weights rather than assuming each image represents all training classes equally. The paper reports that this approach outperforms LSRO across large- and small-scale re-ID datasets.

B. The Same vs. Different Virtual Labels

  • MpRL gives different generated images weight-based virtual labels based on predicted probabilities, addressing ambiguity that can arise when all generated images share one label.The paper illustrates distinct generated images with different clothing colors and says experiments show better performance when assigning different labels.Source: B. The Same vs. Different Virtual Labels · p. 6

C. The Same vs. Different Weights from Pre-defined Training Classes

  • LSRO’s equal class weights are impractical because GAN training uses mini-batches and maps continuous noise to discrete real data, biasing generated-data distributions.The paper argues that a generated image therefore cannot embed equal properties of all pre-defined class distributions.Source: C. The Same vs. Different Weights from… · p. 6
  • MpRL outperforms LSRO on three large- and two small-scale person re-ID datasets by using different weights from pre-defined training classes.The paper says Table I summarizes properties of MpRL and other virtual labels and that MpRL achieves better performance; numerical evidence is presented later.Source: C. The Same vs. Different Weights from… · p. 6

V. EXPERIMENTS

The experiments evaluate MpRL on five person re-ID datasets spanning large and small scales, with Market-1501 and VIPeR highlighted as differing in scale. The section also identifies a comparison table covering properties of four virtual-label approaches.

V. EXPERIMENTS

  • The evaluation uses five datasets: Market-1501, DukeMTMC-reID, CUHK03, VIPeR, and CUHK01; Market-1501 and VIPeR represent different dataset scales.The paper classifies the first three as large-scale and the latter two as small-scale.Source: §V · pp. 6–7
  • Table I compares properties of the all-in-one, one-hot pseudo, LSRO, and proposed MpRL virtual labels.The passage identifies the comparison’s scope but does not provide individual table entries.Source: Table I · p. 7

A. Person Re-ID Datasets

  • Market-1501 has 751 training identities and 12,936 training images, while DukeMTMC-reID uses 702 training identities and 16,522 training images.Market-1501 uses single- and multiple-query settings; DukeMTMC-reID has 2,228 queries and 17,661 gallery images.Source: A. Person Re-ID Datasets · p. 7
  • CUHK03 contains 14,097 images of 1,467 identities, and the experiments use detected boxes with a single-query setting.Each identity is observed from two disjoint camera views; the dataset averages 9.6 training images per identity.Source: A. Person Re-ID Datasets · p. 7
  • VIPeR has 632 identities with two images each; CUHK01 experiments use its more limited 485-identity training split and multiple-query testing.VIPeR divides identities equally between training and testing; CUHK01 contains 3,884 images overall.Source: A. Person Re-ID Datasets · p. 7

B. Experimental Setup

The experiments generate person images with DCGAN and evaluate virtual labels using Identif and Two-stream CNNs. With real images alone, the networks provide rank-1 accuracy baselines across five datasets.

1) GAN Models for Generating Data:

  • DCGAN training uses training-set images resized to 128 × 128, then generates images that are resized to 256 × 256 for CNN training.The generator takes 100-dimensional random noise; training lasts 30 epochs for large datasets and 60 for small datasets.Source: 1) GAN Models for Generating Data · p. 7
  • Although people can readily recognize the generated images as fake, adding them with MpRL improves performance in the experiments.Figure 4 shows generated images in the first two rows and real images in the third row across the five datasets.Source: 1) GAN Models for Generating Data · p. 7, Figure 4 · p. 8

2) CNNs for Evaluation:

  • Identif treats re-ID as classification over real-data training identities, while Two-stream adds a verification function; both use ResNet-50 components.The final fully connected layer has K neurons, and generated images receive MpRL without adding output classes.Source: 2) CNNs for Evaluation · p. 7, Figure 5 · p. 8

C. The CNN Performance

  • With real images only, Identif (Two-stream) rank-1 accuracy is 74.08% (81.83%) on Market-1501, 61.94% (72.62%) on DukeMTMC-reID, 63.10% (81.88%) on CUHK03, 40.76% (51.84%) on VIPeR, and 65.33% (77.78%) on CUHK01.Table II reports Identif and Two-stream performance using only real images; the paper presents these results as the baseline for evaluating MpRL with generated data.Source: C. The CNN Performance · p. 8, Table II · p. 8

D. Generated Data Improve the Performance of The Identif Network

MpRL improves the Identif network when GAN-generated images supplement training across large- and small-scale re-ID datasets. The section compares MpRL implementations and virtual-label alternatives, and examines how generated-image quantity affects results.

D. Generated Data Improve the Performance of The Identif Network

  • +6.29% rank-1 on Market-1501, +6.30% on DukeMTMC-reID, +5.58% on CUHK03, +5.84% on VIPeR, and +3.48% on CUHK01 over the Identif baseline.On the three large-scale datasets, 24,000 generated images were added; VIPeR used 1,200 and CUHK01 used 4,000.Source: D. Generated Data Improve the… · p. 9

E. Comparison with Different Implementations of MpRL

  • On Market-1501, dMpRL-II improves mAP by +5.91% and rank-1 accuracy by +6.29%, exceeding dMpRL-I and sMpRL on both metrics.The authors report that dMpRL-II performs best once the network is relatively stable after 20 training epochs.Source: E. Comparison with Different… · p. 9

F. Comparison with Existing Virtual Labels

  • On Market-1501, dMpRL-II improves mAP and rank-1 by +5.91% and +6.29%; adding 48,000 generated images instead reduces performance.The comparison also reports that multiple-label distributions outperform one-hot distributions in rank-1 accuracy, and that assigning different labels improves results within both distribution types.Source: F. Comparison with Existing Virtual… · pp. 9–10, Table IV · p. 10
  • On VIPeR, dMpRL-II improves mAP by +5.87% and rank-1 by +5.84% with 1,200 generated images, while 12,000 generated images yield inferior results.For small datasets, the authors set generated-image counts to approximately twice the number of real training images.Source: F. Comparison with Existing Virtual… · p. 10
  • Across all five datasets, dMpRL-II outperforms LSRO when different weights from pre-defined training classes are used.The comparison uses the Identif network; 24,000 generated images are used on the three large-scale datasets and 1,200 or 4,000 on the small-scale datasets.Source: F. Comparison with Existing Virtual… · p. 10

G. Comparison with Different GAN Models

The paper finds that generated-image perceptual quality does not significantly affect MpRL's re-ID performance, despite WGAN-GP producing better-looking images than DCGAN. It also reports state-of-the-art comparisons for the Two-stream network with dMpRL-II and concludes that MpRL improves CNN performance using generated data.

G. Comparison with Different GAN Models

H. Comparison with The State-of-the-art Methods

  • With dMpRL-II, the Two-stream network achieves state-of-the-art results on Market-1501, DukeMTMC-reID, CUHK03, and VIPeR; re-ranking further boosts results on the three large-scale datasets.Re-ranking does not further improve performance on the two small-scale datasets.Source: H. Comparison with The… · p. 11, Table VII · p. 11
  • The Two-stream network's results remain below DPFL's rank-1 accuracy on Market-1501 and Spindle's on CUHK01, while exceeding SpindleNet on VIPeR and the three large-scale datasets.The authors compare a single-model Two-stream network with DPFL's ensemble model and note that SpindleNet used nine re-ID datasets for pre-training on the two small datasets, compared with their three.Source: H. Comparison with The… · pp. 11–12

VI. CONCLUSION

  • The paper proposes MpRL as a virtual label for GAN-generated data used in semi-supervised CNN training and reports improvements over LSRO with two CNNs.It identifies extending generated-data virtual labels to other fields as future work.Source: §VI · p. 12

Related papers

Loading 1801.06742v3…