Source-linked AI summary
Person Transfer GAN to Bridge Domain Gap for Person Re-Identification
Longhui Wei, Shiliang Zhang, Wen Gao, Qi Tian
TL;DR
Person ReID needs datasets and methods that handle realistic variation and cross-dataset domain gaps, which limit reuse of labeled training data. The paper introduces the large, realistic MSMT17 dataset and PTGAN for identity-preserving style transfer between domains. Experiments report that PTGAN improves performance over original training data, while large-dataset transfer remains computationally non-optimal.
Problem
Existing ReID datasets simplify realistic conditions, while domain gaps cause severe performance drops and limit reuse of available labeled training data across testing domains.
Method
The paper constructs MSMT17 and proposes PTGAN to transfer persons between datasets while matching target styles and preserving appearance and identity cues.
Results
MSMT17 contains 4,101 identities and 126,441 bounding boxes, and transferred datasets improve GoogLeNet performance over original training sets.
Takeaways & Limitations
The dataset supports more realistic and challenging ReID evaluation, while person transfer can leverage labeled data from other datasets and potentially reduce new annotation costs.
Takeaways & Limitations
For large target datasets, transferring separately to each camera is expensive, so the paper uses one PTGAN across cameras and calls this non-optimal.
Abstract
from arXiv · showhide
Although the performance of person Re-Identification (ReID) has been significantly boosted, many challenging issues in real scenarios have not been fully investigated, e.g., the complex scenes and lighting variations, viewpoint and pose changes, and the large number of identities in a camera network. To facilitate the research towards conquering those issues, this paper contributes a new dataset called MSMT17 with many important features, e.g., 1) the raw videos are taken by an 15-camera network deployed in both indoor and outdoor scenes, 2) the videos cover a long period of time and present complex lighting variations, and 3) it contains currently the largest number of annotated identities, i.e., 4,101 identities and 126,441 bounding boxes. We also observe that, domain gap commonly exists between datasets, which essentially causes severe performance drop when training and testing on different datasets. This results in that available training data cannot be effectively leveraged for new testing domains. To relieve the expensive costs of annotating new training samples, we propose a Person Transfer Generative Adversarial Network (PTGAN) to bridge the domain gap. Comprehensive experiments show that the domain gap could be substantially narrowed-down by the PTGAN.
1. Introduction
Person ReID remains difficult in realistic camera networks because existing datasets simplify real-world variation and models suffer severe cross-dataset domain gaps. The paper introduces MSMT17 and PTGAN to address these limitations.
- Real-world ReID involves indoor and outdoor scenes, long video periods, lighting variation, viewpoint and pose changes, and many identities.
- Training on CUHK03 and testing on PRID yields only 2.0% Rank-1 accuracy, illustrating a severe domain gap.The gap reflects differences in lighting, resolution, human race, seasons, and backgrounds.
- MSMT17 uses a 15-camera indoor/outdoor network, long collection periods, and contains 4,101 identities with 126,441 bounding boxes.These design choices target realistic scene, time, lighting, and scale variation.
- PTGAN transfers persons between datasets while preserving identity and adapting styles such as backgrounds and lighting to the target domain.It adds person-foreground constraints beyond Cycle-GAN to stabilize identities during transfer.
- The paper contributes MSMT17, person transfer for leveraging labeled data across datasets, and analyses of issues hindering practical ReID applications.
2. Related Work
The related work connects person ReID descriptor learning with GAN-based image-to-image translation. It positions PTGAN as an early GAN approach specifically targeting person transfer for ReID.
- Deep learning descriptors outperform hand-crafted features on most person ReID datasets, using classification, verification, and triplet-loss objectives.
- Global descriptors can miss detailed identity cues, motivating local and part-based features alongside complementary hand-crafted representations.
- GAN variants address tasks including style transfer, super-resolution, sketch-to-image generation, and image-to-image translation.
- Conditional adversarial translation requires paired training data, whereas unpaired translation methods address settings where paired images are difficult to acquire.
- The paper characterizes its person-transfer-by-GAN approach as early work differing from prior methods in both motivation and methodology.
3. MSMT17 Dataset
MSMT17 is designed as a larger and more realistic ReID benchmark than prior datasets, combining many cameras, indoor/outdoor scenes, multiple time slots, and extensive annotations. Its protocol uses CMC and mAP for retrieval evaluation.
- Overview of Previous Datasets: Existing datasets are limited by small identity and camera counts, single scenes, short collection periods, and costly or outdated bounding-box generation.
- MSMT17 Construction: MSMT17 collects videos from a 15-camera campus network with 12 outdoor and 3 indoor cameras across four days and morning, noon, and afternoon periods.The collection simulates real scenarios with different weather conditions and long-duration temporal variation.
- Description to MSMT17: 126,441 bounding boxes and 4,101 identities make MSMT17 substantially larger than previous person ReID datasets.
- Description to MSMT17: MSMT17 combines indoor and outdoor videos from 15 locations, producing complex backgrounds and scene variation for ReID.
- Description to MSMT17: 12 time slots across four days produce severe lighting changes while better simulating real scenarios than previous datasets.
- Description to MSMT17: Faster R-CNN is used for bounding-box detection because it is described as easier to implement and more accurate than hand drawing or DPM.
- Evaluation Protocol: The dataset uses a 1:3 training-to-testing ratio, with 32,621 training boxes from 1,041 identities and 93,820 testing boxes from 3,060 identities.
- Evaluation Protocol: ReID performance is evaluated with CMC and mean Average Precision because multiple true positives may be retrieved for each query.
4. Person Transfer GAN
PTGAN transfers person images between ReID datasets by combining unpaired style translation with an identity-preserving foreground constraint. It uses target-domain image styles, foreground masks, and segmentation to generate visually adapted samples while maintaining person appearance.
- PTGAN simulates capturing persons from dataset A with cameras from dataset B, transferring backgrounds, lighting, resolutions, and other dataset-specific styles.The transfer is designed to preserve identity while making source images consistent with the target domain.
- The model combines style transfer with person identity keeping because transferred samples of one person retain the same ReID label.The style and identity objectives are balanced by λ1.
- PTGAN uses Cycle-GAN for unpaired style mapping between datasets, with discriminators for domains A and B and adversarial and cycle-consistency losses.ReID datasets lack paired images of the same person across datasets, motivating the unpaired formulation.
- The identity loss measures changes in the person foreground before and after transfer using a foreground mask M(a).The formulation includes source and target-domain samples, and G(a) denotes the transferred image from a.
- PTGAN can use unlabeled target-domain person images and is therefore suited to new testing domains with limited or no labeled training data.PSPNet extracts person masks; the paper reports that PTGAN remains effective despite noisy segmentation results.
- On CUHK03-to-PRID-cam1 examples, PTGAN transfers PRID camera style while maintaining person appearance and generating similar shadows, road marks, and backgrounds.Figure 4 compares PTGAN and Cycle-GAN outputs, with PSPNet segmentation shown in the second row and person regions marked pink.
5. Experiments
Experiments evaluate MSMT17’s difficulty and PTGAN’s ability to reduce domain gaps across small and large target datasets. Results show lower performance on MSMT17 than established benchmarks, substantial gains after transfer, and annotation-cost savings, while large-dataset transfer remains nonoptimal.
- Performance on MSMT17: MSMT17 yields substantially lower performance than established benchmarks: the baseline reaches 23% mAP versus 51.7% on Market, while GLAD reaches 34% versus 73.9% on Market.PDC and GLAD outperform the baseline by using extra part and regional features, but remain below their reported performance on other datasets.
- Performance on MSMT17: MSMT17 retrieval results contain false positives with similar appearance and true positives with different lighting, poses, and backgrounds.These samples are presented as evidence that the task is challenging while remaining realistic.
- Transfer from Large Dataset to Small Dataset: Rank-1 accuracy on PRID improves from 2.0% to 37.5% for CUHK03 transfer and from 5.0% to 33.5% for Market transfer.For CUHK03 transfer, Rank-10 accuracy also improves from 11.5% to 72.5%.
- Transfer Settings: Combining transferred datasets from two PRID cameras produces better performance, while person transfer to each target camera performs better than a single mapping across cameras.The large-dataset setup does not distinguish individual cameras because transferring separately to many cameras is expensive.
- Performance of Person Transfer on MSMT17: Rank-1 accuracy on MSMT17 improves by 4.7%, 6.8%, and 3.7% after transfer from Duke, Market, and CUHK03, respectively.The authors report that these improvements effectively narrow the domain gaps between MSMT17 and the three source datasets.
- Performance of Person Transfer on MSMT17: 16,522 transferred images from Duke achieve similar Rank-1 performance to 2,602 annotated MSMT17 images, estimated at 6.3 transferred images per annotated image.The comparison uses approximately 11.5%–11.8% Rank-1 accuracy.
6. Conclusions and Discussions
The paper contributes MSMT17 as a realistic, challenging person ReID dataset and proposes PTGAN to bridge dataset domain gaps. Experiments support domain-gap reduction, but the transfer strategy is not yet optimal when target datasets contain cameras with different styles.
- MSMT17 is a large-scale dataset with substantial variation in lighting, scenes, backgrounds, and human poses, defining a more realistic and challenging person ReID task.
- PTGAN transfers persons between datasets to bridge domain gaps, but one mapping function is not optimal for target datasets whose cameras have different styles.The paper identifies camera-style-aware mapping as a direction for more stable and efficient future transfer strategies.