Source-linked AI summary
Joint Discriminative and Generative Learning for Person Re-identification
Zhedong Zheng, Xiaodong Yang, Zhiding Yu, Liang Zheng, Yi Yang, Jan Kautz
TL;DR
Person re-identification remains difficult because camera changes create substantial intra-class variation, while prior generative pipelines are relatively separate from discriminative learning. DG-Net couples both processes end-to-end through shared appearance encoding and code-switched image generation, with generated images fed back online. The framework consistently improves image-generation quality and re-id accuracy across three benchmarks.
Problem
Person re-identification must learn representations robust to substantial intra-class variation across cameras, but existing generative augmentation pipelines remain relatively separate from discriminative re-id learning.
Method
DG-Net jointly optimizes generative and discriminative modules, decomposing images into appearance and structure codes while sharing the appearance encoder and feeding generated images back online.
Results
Experiments on three benchmarks show substantial improvements to both image-generation quality and re-id accuracy, with 8.3% and 10.3% mAP gains on Market-1501 and DukeMTMC-reID over separately generated images.
Takeaways & Limitations
The unified online interaction lets re-id learning better leverage generated data rather than simply training on offline synthesized images.
Takeaways & Limitations
Data bias causes the generative module to learn regular textures while ignoring some rare patterns, such as shirt logos.
Abstract
from arXiv · showhide
Person re-identification (re-id) remains challenging due to significant intra-class variations across different cameras. Recently, there has been a growing interest in using generative models to augment training data and enhance the invariance to input changes. The generative pipelines in existing methods, however, stay relatively separate from the discriminative re-id learning stages. Accordingly, re-id models are often trained in a straightforward manner on the generated data. In this paper, we seek to improve learned re-id embeddings by better leveraging the generated data. To this end, we propose a joint learning framework that couples re-id learning and data generation end-to-end. Our model involves a generative module that separately encodes each person into an appearance code and a structure code, and a discriminative module that shares the appearance encoder with the generative module. By switching the appearance or structure codes, the generative module is able to generate high-quality cross-id composed images, which are online fed back to the appearance encoder and used to improve the discriminative module. The proposed joint learning framework renders significant improvement over the baseline without using generated data, leading to the state-of-the-art performance on several benchmark datasets.
1. Introduction
Person re-identification must handle substantial camera-induced intra-class variation, motivating generative augmentation. DG-Net jointly couples generation and re-id learning so generated images refine the shared appearance encoder online.
- Person re-identification retrieves the same person across non-overlapping cameras despite changes in background, viewpoint, and pose.
- Generative augmentation seeks realistic and diverse images that cover unseen intra-class variations while limiting the domain gap from real data.
- DG-Net decomposes pedestrian images into appearance and structure codes, sharing the appearance encoder between generation and discriminative re-id learning.
- Switching appearance or structure codes generates realistic, diverse intra- and cross-identity compositions without requiring auxiliary pose data.
- Keeping appearance fixed while varying structure changes pose, viewpoint, or background, enabling dynamic soft labels and fine-grained identity learning.
- Experiments report consistent re-id gains over competing methods across several benchmarks and favorable image-generation quality.
2. Related Work
Related re-id work uses metric learning, auxiliary human structure information, GAN augmentation, and style transfer. DG-Net instead combines generative and discriminative learning while avoiding extra pose or parsing inputs.
- Metric-learning methods combine identification, verification, or triplet objectives, while other approaches add pedestrian attributes through multitask learning.
- Structure-aware approaches use pose estimation, human parsing, alignment, or part matching to improve local feature learning and spatial matching.
- DG-Net differs by jointly coupling generation and re-id learning while requiring no auxiliary pose or human-parsing information for image generation.
- GAN-based augmentation includes unconditional generation, pseudo-labeling, shared re-id and discriminator weights, and pose-conditioned image synthesis.
- Style-transfer methods synthesize pedestrian appearances across datasets, camera styles, illumination conditions, or foreground masks.
3. Method
DG-Net jointly trains generation and re-identification by coupling a code-swapping generative module with a discriminative module that shares the appearance encoder. Generated images are used online for dynamic soft-label learning and fine-grained identity mining.
- DG-Net jointly couples image generation and re-identification, feeding self-identity and cross-identity synthetic images online into discriminative learning.The generative and discriminative modules are co-designed rather than trained as separate stages.
- The generative module encodes each image into appearance and structure codes, then decodes code-swapped pairs into composed pedestrian images.Appearance codes retain identity-related semantics, while structure codes preserve geometry and positional information.
- Self-identity generation reconstructs images and same-identity cross-image pairs, while identification loss encourages appearance codes of different identities to remain separable.Same-identity reconstruction reduces intra-class feature variation; identification loss uses the appearance code for identity prediction.
- Cross-identity generation reconstructs latent appearance and structure codes because different-identity composed images lack pixel-level ground-truth supervision.The generated image is constrained through latent-code reconstruction rather than direct image matching.
- Dynamic soft labeling trains primary features by matching the discriminative module’s predictions to teacher-generated probability distributions for composed images.The approach uses a baseline CNN teacher because each synthetic image combines visual content from two real images.
- Fine-grained feature mining uses generated clothing changes to emphasize identity attributes independent of clothing, including hair, hats, bags, and body size.This provides additional identity supervision without manually labeled pedestrian attributes or explicit hard-sample searches.
- The encoders, decoder, discriminator, and shared appearance encoder are jointly optimized with a weighted sum of reconstruction, identification, primary-learning, and fine-grained losses.Image reconstruction, latent-code reconstruction, and discriminative objectives are combined in the total training objective.
4. Experiments
Experiments on Market-1501, DukeMTMC-reID, and MSMT17 evaluate DG-Net's image generation and re-id learning. DG-Net produces realistic, diverse images and consistently improves re-id performance, with joint training outperforming offline generation and competing methods.
- Experimental Setup: DG-Net is evaluated on Market-1501, DukeMTMC-reID, and MSMT17 using qualitative and quantitative comparisons for generative and discriminative performance.The experiments compare image generation and re-id accuracy against state-of-the-art methods.
- Qualitative Generative Evaluation: Online feeding and identity supervision improve generation quality, whereas removing them causes blurry edges, undesired textures, and inaccurate clothing appearance.The ablations show that joint discriminative learning benefits image generation.
- Qualitative Generative Evaluation: DG-Net's generated images are more realistic than those from LSGAN, FD-GAN, PG2-GAN, and PN-GAN across foreground and background regions.LSGAN produces artifacts and duplicated patterns, FD-GAN blurry images, while the pose-conditioned methods retain blurs and background artifacts.
- Generative Analysis: Appearance-code interpolation produces continuous representations that generalize beyond memorized visual patterns, while structure-code interpolation is less realistic.The complementary structure interpolation keeps appearance codes fixed and changes structural information.
- Quantitative Generative Evaluation: DG-Net significantly outperforms other methods on both FID and SSIM, indicating improved realism and diversity from code switching.SSIM exceeds that of the original training set because structure-code switching introduces varied poses, carryings, and backgrounds.
- Discriminative Evaluation: Combining primary and fine-grained features improves over baseline by 6.1% for Rank@1 and 12.4% for mAP across three datasets.End-to-end training reaches 86.0% mAP on Market-1501 versus 84.4% with independently learned features after offline generation.
5. Conclusion
The proposed framework end-to-end couples re-id learning and image generation in one network, with an online interaction between modules. Across three benchmarks, it improves both image generation quality and re-id accuracy.
- DG-Net end-to-end couples re-id learning and image generation in a unified network.
- An online interactive loop lets the discriminative and generative modules mutually benefit the two tasks.
- Experiments on three benchmarks show substantial improvements in both image generation quality and re-id accuracy.
Appendix
The appendix collects architectural details, additional re-id evaluations, analyses of the learned spaces and feature-learning methods, and structure-code interpolation examples.
- Section A summarizes DG-Net’s architecture details.
- Section B presents additional re-id evaluations.
- Section C gives further rationales for the appearance and structure spaces and primary and fine-grained feature learning on appearance codes.
- Section D demonstrates image generation by interpolating between structure codes.
A. Network Architectures
DG-Net uses appearance and structure encoders, a decoder, and a discriminator built primarily from convolutional and residual components, with normalization and multi-scale processing details described.
- DG-Net consists of appearance and structure encoders, decoder G, and discriminator D.
- The modules mainly use convolutional layers and residual blocks, following common GAN construction practices.
- The structure encoder applies instance normalization and LReLU after convolutional layers, with optional ASPP for multi-scale features.
- Decoder G uses residual blocks followed by upsampling and convolutional layers.
B. More Discriminative Evaluations
On the relatively small CUHK03-NP dataset, DG-Net achieves 65.6% Rank@1 and 61.1% mAP.
- 65.6% Rank@1 is achieved by DG-Net on CUHK03-NP.CUHK03-NP contains much fewer images than the other listed benchmarks, averaging 9.6 training images per person.
- 61.1% mAP is achieved by DG-Net on CUHK03-NP.The evaluation is presented as a further assessment on a relatively small dataset.
C. Appearance and Structure Codes
DG-Net separates pedestrian representations into appearance and structure codes, while discriminative learning recovers identity-related information that structure cues may contain. Its architecture components support image synthesis and complementary feature learning, with transfer and benchmark evaluations included.
- Architecture: The structure encoder, decoder, and discriminator are specified as separate architectural components in Tables 6–8.The supplied passages identify these tables as architecture descriptions for Es, G, and D.
- Code roles: Appearance codes primarily encode appearance and identity-related semantics, whereas structure codes preserve geometry, position, pose, background, and other variations for synthesis.The structure encoder is designed to preserve low-level positional and geometric information, although some structure cues remain identity-related.
- Evaluation: DG-Net is evaluated with direct transfer learning and compared with state-of-the-art results on CUHK03-NP.The supplied table captions identify Table 9 as direct transfer results and Table 10 as CUHK03-NP comparisons.
- Feature learning: Fine-grained feature mining extracts identity-related semantics from structure cues that softmax learning may otherwise overlook.This promotes complementary primary and fine-grained features rather than relying only on appearance information.
D. Interpolate between Structure Codes
Interpolating structure codes while fixing appearance codes produces mostly unrealistic foregrounds and backgrounds, contrasting with the continuity observed for appearance-code interpolation.
- Structure-code interpolation: Most images generated by interpolating between structure codes look unrealistic in both foreground and background regions.The experiment keeps appearance codes intact while linearly interpolating structure codes.
- Structure-code interpolation: Figure 9 fixes the appearance code in each row and illustrates image generation through linear interpolation of two structure codes.It is intended for zoomed comparison with Figure 5.
E. Direct transfer learning
The direct transfer protocol trains DG-Net on one dataset and tests it on another without adaptation; Table 9 reports the resulting generalizability evaluation.
- Protocol: Direct transfer learning trains the model on dataset A and tests it directly on dataset B with no adaptation.This protocol is denoted A→B.
- Results: Table 9 presents the direct transfer learning results for this cross-dataset evaluation.