Source-linked AI summary
FD-GAN: Pose-guided Feature Distilling GAN for Robust Person Re-identification
Yixiao Ge, Zhuowan Li, Haiyu Zhao, Guojun Yin, Shuai Yi, Xiaogang Wang, Hongsheng Li
TL;DR
Person re-identification needs robust identity features despite pose variation, but existing approaches may require pose information and added inference cost. FD-GAN uses Siamese feature learning with pose-guided generation, identity and pose discriminators, and same-pose loss. It reports state-of-the-art performance on three re-identification datasets without auxiliary pose information or additional inference cost.
Problem
Pose variation challenges robust person re-identification, while alignment and human-region methods generally require auxiliary pose information and increased inference computation.
Method
FD-GAN uses a Siamese encoder-generator framework with identity verification, identity and pose discriminators, and a same-pose loss to learn identity-related, pose-unrelated features.
Results
FD-GAN achieves state-of-the-art person re-identification performance on Market-1501, CUHK03, and DukeMTMC-reID without additional inference cost or pose information.
Takeaways & Limitations
The learned pose-unrelated representations support robust re-identification while avoiding auxiliary pose information and additional computational cost during inference.
Takeaways & Limitations
The learned encoder may focus on overall appearance while failing to capture distinguishable appearance details in some generated images.
Abstract
from arXiv · showhide
Person re-identification (reID) is an important task that requires to retrieve a person's images from an image dataset, given one image of the person of interest. For learning robust person features, the pose variation of person images is one of the key challenges. Existing works targeting the problem either perform human alignment, or learn human-region-based representations. Extra pose information and computational cost is generally required for inference. To solve this issue, a Feature Distilling Generative Adversarial Network (FD-GAN) is proposed for learning identity-related and pose-unrelated representations. It is a novel framework based on a Siamese structure with multiple novel discriminators on human poses and identities. In addition to the discriminators, a novel same-pose loss is also integrated, which requires appearance of a same person's generated images to be similar. After learning pose-unrelated person features with pose guidance, no auxiliary pose information and additional computational cost is required during testing. Our proposed FD-GAN achieves state-of-the-art performance on three person reID datasets, which demonstrates that the effectiveness and robust feature distilling capability of the proposed FD-GAN.
1 Introduction
Person re-identification must handle pose variation, blur, and occlusion while matching identities across cameras. FD-GAN learns pose-unrelated identity representations using pose-guided generation and discriminators, without requiring pose information or added inference cost.
- Person re-identification matches pedestrian images of the same identity across multiple cameras despite pose variation, blur, and occlusion.
- Existing alignment and body-region methods require auxiliary pose information during inference and increase computational cost through pose estimation.
- FD-GAN uses a Siamese structure with image encoders and generators to learn identity-related, pose-unrelated representations.
- Identity and pose discriminators, verification, reconstruction, and same-pose losses jointly regularize feature learning and mitigate pose- and background-related information.
- FD-GAN achieves state-of-the-art re-identification performance on Market-1501, CUHK03, and DukeMTMC-reID without extra pose information or inference complexity.
2 Related Work
Prior re-identification research uses GANs for image synthesis, regularization, domain-gap reduction, and representation learning. FD-GAN instead targets pose-invariant identity representations through adversarial feature learning.
- GANs combine generators and discriminators, with the generator producing images and the discriminator distinguishing generated from real distributions.
- The framework uses a Siamese structure and same-pose loss to encourage robust identity-related and pose-unrelated features.
- FD-GAN aims to decompose pose information from image features through adversarial training to learn identity-related and pose-unrelated representations.
- Previous re-identification methods commonly learned discriminative representations or used metric learning, while some incorporated alignment or body-region pose information.
- Earlier GAN-based re-identification methods addressed unlabeled-data regularization and domain gaps between separate datasets.
3 Feature Distilling Generative Adversarial Network
FD-GAN learns identity-related and pose-unrelated person representations through a Siamese encoder–generator framework guided by identity and pose discrimination. Its training combines verification, adversarial, reconstruction, and same-pose objectives, while inference requires no auxiliary pose information.
- FD-GAN targets identity-related and pose-unrelated representations for person re-identification under large pose variations.
- The Siamese framework uses image encoders, generators, an identity verification classifier, and identity and pose discriminators.Each branch receives a person image and target pose landmark map; the encoder produces features and the generator synthesizes a target-pose image.
- The verification classifier predicts whether two encoded person images share an identity using feature differences and a binary cross-entropy objective.It processes element-wise subtraction and squaring, followed by batch normalization, a fully connected layer, and sigmoid output; C=1 denotes the same person and C=0 otherwise.
- Identity and pose discriminators regularize generated images to preserve identity and match the target pose, while online pose-map augmentation reduces pose overfitting.The identity discriminator compares generated and input images, whereas the pose discriminator evaluates image–pose correspondence.
- The same-pose loss compares two generated images of one person under the same target pose, encouraging features to retain identity information while ignoring other factors.
- The overall objective combines verification, identity-discriminator, pose-discriminator, reconstruction, and same-pose losses, with alternating optimization across three training stages.Reconstruction minimizes L1 differences from corresponding real images and supports more stable generator convergence; auxiliary-loss weights are λid, λpd, λr, and λsp.
- FD-GAN differs from DR-GAN through its Siamese same-pose design, unshared encoder and identity-discriminator weights, and verification classifier.
4 Experiments
Experiments evaluate FD-GAN on three reID datasets, analyze its components, compare it with prior methods, and examine generated images and learned features. FD-GAN achieves strong reID performance while its components improve representation learning and generation quality, though visualizations reveal missing appearance details.
- Component analysis: The Siamese baseline outperforms the single-branch baseline by 12.7% and 20.6% in mAP on the two evaluated datasets.The comparison isolates the contribution of the Siamese structure.
- Component analysis: FD-GAN improves over the Siamese baseline by 5.2% and 3.2% in mAP on the two datasets, with gains in both mAP and top-1 accuracy.Removing online pose-map augmentation causes a 0.5% performance drop on both datasets.
- Component analysis: Removing the identity or pose discriminators causes performance drops and poorer generated images.The component analysis evaluates the discriminators separately and together.
- Comparison with state-of-the-arts: FD-GAN achieves 90.5% top-1 accuracy and 77.7% mAP on Market-1501, 92.6% and 91.3% on CUHK03, and 80.0% and 64.5% on DukeMTMC-reID.The reported metrics are top-1 accuracy followed by mAP for each dataset.
- Person image generation and visual analysis: Compared with prior person-generation methods, FD-GAN better captures backpack semantics and generates more correct upper- and lower-body clothing.The paper attributes the result to identity supervision, the Siamese structure, and same-pose loss.
- Person image generation and visual analysis: A visualization example shows that generated frontal images preserve general colors and body shapes but omit a distinguishable upper-body colored pattern.The authors use generated images to inspect which aspects of person appearance the learned features capture.
5 Conclusion
The paper concludes that FD-GAN learns identity-related and pose-unrelated person representations using human pose guidance. It reports state-of-the-art reID performance without extra pose information or inference cost, alongside higher-quality generated images.
- FD-GAN learns identity-related and pose-unrelated person representations with human pose guidance for robust person reID.Its Siamese structure and losses are intended to learn more pose-invariant features.
- The framework achieves state-of-the-art person reID without additional computational cost or extra pose information during inference.The conclusion also reports higher-quality generated person images than existing specific person-generation methods.
- FD-GAN's generated person images show higher quality than those from existing specific person-generation methods.