Source-linked AI summary
Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in vitro
Zhedong Zheng, Liang Zheng, Yi Yang
TL;DR
The paper addresses how to generate and use additional training data from an existing labeled set for semi-supervised representation learning. It generates unlabeled images with a GAN and integrates them into CNN training using LSRO, reporting consistent improvements over baseline systems across person re-ID and fine-grained recognition.
Problem
The paper addresses using GAN-generated samples in existing learning frameworks and combining labeled and unlabeled data without extra data collection.
Method
The method trains a GAN on the original set, then uses LSRO to assign generated unlabeled images a uniform label distribution while training the CNN.
Results
The proposed pipeline shows consistent performance improvement over re-ID and fine-grained recognition baselines.
Takeaways & Limitations
Imperfect GAN images can provide regularization when mixed with labeled real images for simultaneous semi-supervised representation learning.
Takeaways & Limitations
The paper provides baseline evaluations with imperfect GAN images and leaves the effect of better visual quality for future investigation.
Abstract
from arXiv · showhide
The main contribution of this paper is a simple semi-supervised pipeline that only uses the original training set without collecting extra data. It is challenging in 1) how to obtain more training data only from the training set and 2) how to use the newly generated data. In this work, the generative adversarial network (GAN) is used to generate unlabeled samples. We propose the label smoothing regularization for outliers (LSRO). This method assigns a uniform label distribution to the unlabeled images, which regularizes the supervised model and improves the baseline. We verify the proposed method on a practical problem: person re-identification (re-ID). This task aims to retrieve a query person from other cameras. We adopt the deep convolutional generative adversarial network (DCGAN) for sample generation, and a baseline convolutional neural network (CNN) for representation learning. Experiments show that adding the GAN-generated data effectively improves the discriminative ability of learned CNN embeddings. On three large-scale datasets, Market-1501, CUHK03 and DukeMTMC-reID, we obtain +4.37%, +1.6% and +2.46% improvement in rank-1 precision over the baseline CNN, respectively. We additionally apply the proposed method to fine-grained bird recognition and achieve a +0.6% improvement over a strong baseline. The code is available at https://github.com/layumi/Person-reID_GAN.
1. Introduction
The paper proposes an in vitro semi-supervised pipeline that generates unlabeled samples from the original training set with a GAN and integrates them into CNN learning through LSRO. It targets limited person re-ID data and reports consistent improvements over CNN baselines across re-ID and fine-grained recognition.
- Motivation and approach: The pipeline expands the original labeled training set with GAN-generated unlabeled images for semi-supervised CNN learning.The method uses generated samples without an additional data collection process.
- Motivation and approach: LSRO assigns GAN-generated images a uniform label distribution over existing classes instead of assigning them known-class labels.This treats generated samples as outliers and avoids forcing them into a particular training class.
- Motivation and approach: Person re-ID remains constrained by limited images per identity, averaging 17.2 in Market-1501, 9.6 in CUHK03, and 23.5 in DukeMTMC-reID.The task also requires costly pedestrian bounding-box and identity annotation.
- Contributions: LSRO is reported as superior to two available strategies for handling unlabeled data.The comparison concerns the “All in one” and “Pseudo label” strategies.
- Contributions: The pipeline consistently improves a ResNet baseline on three person re-ID datasets and one fine-grained recognition dataset.The supplied introduction identifies this as a central empirical contribution.
2. Related Work
This section introduces the paper’s discussion of prior work on GANs, semi-supervised learning, and person re-ID.
- The related-work section covers GANs, semi-supervised learning, and person re-identification.
2.2. Semi-supervised Learning
Semi-supervised learning incorporates unlabeled data when annotated data are limited, while prior approaches use auxiliary unsupervised objectives or CNN-based feature-learning strategies.
- Semi-supervised learning: Semi-supervised learning considers unlabeled data, especially when the volume of annotated data is small.
- Semi-supervised learning: Prior work uses unsupervised learning as an auxiliary task, including pre-training, input reconstruction, and representation denoising.
- Person re-identification: The paper positions its approach within semi-supervised representation learning rather than explicit part-matching alone.
- Person re-identification: Person re-ID research has advanced from handcrafted features toward CNNs using part-based, global, gated, and identification-based representations.
3. Network Overview
The network overview combines a GAN trained on real training data with a CNN trained on real and generated images. The CNN keeps a K-class output and assigns generated samples uniform labels rather than treating them as an extra class.
- Pipeline: Real training images first train the GAN, after which real and generated samples are combined as CNN training input.
- Generative adversarial model: The GAN generator starts from a 100-dimensional random vector and uses deconvolutional layers with rectified linear units and batch normalization.
- Generative adversarial model: The discriminator receives generated and real training images and classifies them as fake or real using convolutional layers and a fully connected layer.
- Convolutional neural network: The ResNet-50 CNN simultaneously trains on labeled and unlabeled images, with generated images resized to 256 × 256 × 3.
- Convolutional neural network: The CNN output remains K-dimensional because generated samples receive a uniform distribution over existing classes rather than an extra-class label.
4. The Proposed Regularization Method
The paper revisits label smoothing and extends it to GAN-generated outliers through LSRO, assigning unlabeled images a uniform distribution over training classes. Real images retain one-hot labels, while generated images use a separate loss that regularizes confidence and adds training variation.
- 4.1. Label Smoothing Regularization Revisit: Label smoothing regularization (LSR) assigns small probabilities to non-ground-truth classes, discouraging overconfidence and reducing over-fitting.The method is formulated with cross-entropy loss and typically uses ε = 0.1.
- 4.2. Label Smoothing Regularization for Outliers: LSRO extends label smoothing to unlabeled GAN-generated images by assigning a uniform label distribution over all training classes.The generated samples are assumed not to belong to any predefined class, so the network is discouraged from predicting one class with high confidence.
- 4.2. Label Smoothing Regularization for Outliers: Real images use the conventional one-hot distribution, whereas generated images use LSRO in a combined cross-entropy objective.The system therefore has separate losses for real and generated images, controlled by Z = 0 for real data and Z = 1 for generated data.
- 4.2. Label Smoothing Regularization for Outliers: LSRO introduces additional color, lighting, and pose variation by incorporating generated outliers near real training images.This is intended to prevent the model from treating incidental features, such as one identity’s clothing color, as discriminative identity cues.
- 4.2. Label Smoothing Regularization for Outliers: LSRO is compared with assigning generated samples a new class and with dynamically assigning pseudo-labels during CNN training.The reported experiments find LSRO superior to both “All in one” and “Pseudo label.”
5. Experiment
Experiments evaluate LSRO with GAN-generated images across person re-identification and fine-grained bird recognition. The method consistently improves CNN baselines, while results also examine data quantity, competing integration strategies, and dataset transfer.
- Experimental setup: The experiments cover Market-1501, CUHK03, DukeMTMC-reID, and CUB-200-2011 using ResNet CNN and DCGAN-generated data.Market-1501 is the main evaluation dataset; CUHK03, DukeMTMC-reID, and CUB-200-2011 provide additional tests.
- Baselines: The ResNet baseline achieves rank-1 accuracy of 73.69% on Market-1501, 71.5% on CUHK03, and 60.28% on DukeMTMC-reID.These baseline results are reported as comparable to prior results.
- Person re-identification: +4.37% rank-1 accuracy and +4.75% mAP on Market-1501 result from adding 24,000 GAN images with LSRO.The reported rank-1 change is from 73.69% to 78.06%, while mAP changes from 51.48% to 56.23%.
- Person re-identification: +1.6% rank-1 accuracy on CUHK03 and +2.46% rank-1 accuracy on DukeMTMC-reID improve the corresponding ResNet baselines.CUHK03 also improves in rank-5, rank-10 accuracy, and mAP; DukeMTMC-reID mAP improves by +2.14%.
- Data quantity: Adding different numbers of GAN images consistently improves the baseline, but peak performance occurs with 2×GAN images because too many samples can drive uniform predictions.Approximately 3×GAN images still produce a +2.38% improvement in rank-1 accuracy, indicating a quantity trade-off.
- Regularization analysis: Real CUHK03 images also improve regularization, while GAN-generated images perform slightly better despite being visually imperfect.The comparison uses 12,000 real CUHK03 images versus 12,000 GAN images.
- Comparison methods: LSRO exceeds “All in one” and “Pseudo label” by approximately +1% ∼+2% on Market-1501.The paper attributes this to LSRO making a less strong assumption about the labels of GAN images.
- Fine-grained recognition: +0.6% recognition accuracy on CUB-200-2011 improves the baseline from 82.6% to 83.2%.An ensemble of two models reaches 84.4%; the paper focuses on regularization rather than a state-of-the-art result.
6. Conclusion
The paper demonstrates that imperfect GAN-generated images can regularize representation learning when mixed with labeled real data through LSRO. It reports consistent improvements over re-ID and fine-grained recognition baselines, while leaving the effect of GAN image quality for future study.
- GAN-generated images demonstrate regularization ability when trained with a ResNet baseline model.The paper describes this as an “in vitro” use of GANs for representation learning.
- LSRO mixes unlabeled GAN images with labeled real training images for simultaneous semi-supervised learning.
- The method delivers consistent performance improvement over re-ID and fine-grained recognition baseline systems.
- The effect of using better-quality GAN images in supervised learning remains an open question.The reported evaluations use imperfect GAN images, and the authors identify quality-dependent performance as future work.