Source-linked AI summary
Identity-Guided Human Semantic Parsing for Person Re-Identification
Kuan Zhu, Haiyun Guo, Zhiwei Liu, Ming Tang, Jinqiao Wang
TL;DR
Existing alignment-based person re-ID methods rely on pretrained parsing models for pixel-level alignment and omit personal belongings. ISP learns pixel-level body-part and belonging semantics from identity labels through iterative feature-map clustering, and achieves state-of-the-art performance on three person re-ID datasets.
Problem
Pretrained parsing models used for pixel-level alignment can exclude personal belongings that are useful for person re-ID.
Method
ISP iteratively clusters feature-map pixels from identity-matched images to create pseudo-labels, learn part estimation, and pool visible body-part and belonging features.
Results
ISP sets new state-of-the-art performance on Market-1501, DukeMTMC-reID, and CUHK03-NP.
Takeaways & Limitations
Identity supervision can support pixel-level alignment of human parts and personal belongings while matching only shared-visible parts.
Abstract
from arXiv · showhide
Existing alignment-based methods have to employ the pretrained human parsing models to achieve the pixel-level alignment, and cannot identify the personal belongings (e.g., backpacks and reticule) which are crucial to person re-ID. In this paper, we propose the identity-guided human semantic parsing approach (ISP) to locate both the human body parts and personal belongings at pixel-level for aligned person re-ID only with person identity labels. We design the cascaded clustering on feature maps to generate the pseudo-labels of human parts. Specifically, for the pixels of all images of a person, we first group them to foreground or background and then group the foreground pixels to human parts. The cluster assignments are subsequently used as pseudo-labels of human parts to supervise the part estimation and ISP iteratively learns the feature maps and groups them. Finally, local features of both human body parts and personal belongings are obtained according to the selflearned part estimation, and only features of visible parts are utilized for the retrieval. Extensive experiments on three widely used datasets validate the superiority of ISP over lots of state-of-the-art methods. Our code is available at https://github.com/CASIA-IVA-Lab/ISP-reID.
1 Introduction
Person re-ID suffers from pixel-level misalignment and appearance changes caused by occlusion, pose, detection, and viewpoint variation. ISP addresses these limitations by learning pixel-level semantics for body parts and personal belongings using only identity labels.
- Misalignment from occlusion, inaccurate detection, pose variation, and viewpoint changes makes person re-ID retrieval difficult.
- Extra semantic methods depend on pretrained parsing or pose models that can be error-prone and discard personal belongings as background.Backpacks and reticules may provide useful contextual cues for identifying a person.
- ISP uses cascaded clustering on feature maps to generate pseudo-labels for human parts from identity-labeled images.Pixels are first grouped into foreground or background, then foreground pixels from images sharing an identity are clustered into parts.
- ISP iteratively refines pseudo-labels and part estimation, then pools local features for body parts and personal belongings while matching only shared-visible parts.The approach is applicable across popular backbone networks.
- ISP reports new state-of-the-art performance on Market-1501, DukeMTMC-reID, and CUHK03-NP.
2 Related work
Prior alignment methods range from rigid or learned region alignment to attention and extra semantic cues, but remain coarse, inconsistent, or dependent on pretrained models. ISP instead clusters pixels into semantically consistent parts using identity labels.
- Weakly supervised semantic segmentation methods are not directly suitable for end-to-end weakly supervised human parsing in person re-ID.They target different levels and use complex structures and objectives.
- Rigid stripe methods partition images horizontally, but their coarse regions introduce background noise and poorly align human parts.
- Auto-localization methods learn latent grids, but these grids can overlap, remain coarse, and use a fixed number of parts that cannot handle occlusion.
- Attention methods suppress background and enhance discriminative regions without explicitly locating semantic parts or guaranteeing cross-image focus consistency.
- Extra semantic methods achieve pixel-level alignment using pretrained part or pose models, but their utility and robustness depend on semantic estimation accuracy and they miss personal belongings.
- ISP groups pixels into human parts to generate pseudo-part labels with identity supervision while enforcing semantic consistency across images.The method applies clustering to end-to-end person re-ID rather than only image-level classification or unsupervised re-ID.
3 Methodology
ISP learns pixel-level semantic parts from identity labels through cascaded clustering and iteratively uses them for aligned person re-ID. Matching combines always-present global and foreground features with features from parts visible in both images.
- Pseudo-part-label generation: ISP generates pseudo-part labels by clustering feature-map pixels, first separating foreground from background and then grouping foreground pixels into semantic parts.The approach treats personal belongings as part of the foreground semantic representation.
- Pseudo-part-label generation: Clustering foreground pixels across all images sharing an identity makes part assignments semantically consistent and robust to occlusion.A single occluded image may contain fewer than K−1 semantic parts because clustering uses samples from all images of that person.
- Part-aligned representation learning: The network iteratively refines feature maps and pseudo-labels, using the assignments to supervise part estimation and produce increasingly accurate part representations.The overview repeats pseudo-part-label generation and part-aligned representation learning until convergence.
- Part-aligned representation learning: Part features are obtained by probability-weighted pooling of feature maps, while global and foreground features are separately produced by global average pooling.The local part representations concatenate all part features except background; foreground and global representations are retained directly.
- Aligned representation matching: During matching, ISP combines global and foreground distances with distances between partial features for parts visible in both probe and gallery images.Global and foreground features always exist, whereas local features are restricted to shared-visible parts; the distance metric is cosine distance.
- Aligned representation matching: ISP explicitly addresses occlusion without extra semantic or pose information by identifying visible parts at pixel-level and excluding occluded parts from matching.The method is presented as the first extra semantic free method to explicitly address the occlusion problem.
4 Experiments
ISP is evaluated on holistic and occluded person re-ID benchmarks, with experiments covering implementation, comparisons, parsing quality, visualizations, and ablations. Results show strong performance, learned semantic parts, and benefits from cascaded clustering and aligned matching.
- Holistic re-ID results: ISP achieves the best performance on DukeMTMC-reID, outperforming other methods by at least 0.5% in Rank-1 and 1.6% in mAP.The authors attribute the result partly to the error-proneness of pretrained semantic extraction on this dataset.
- Holistic re-ID results: ISP achieves the best mAP and second-best Rank-1 on Market-1501, with a larger improvement in mAP than Rank-1.The authors relate this pattern to mAP accounting for all target-image ranking positions.
- Holistic re-ID results: On CUHK03-NP, ISP achieves the second-best results and outperforms all methods except DSA-reID despite learning pixel-level semantics without extra supervision.ISP requires every semantic part to appear at least once for a person to ensure high semantic consistency.
- Occluded re-ID results: ISP sets the state of the art on Occluded-DukeMTMC, outperforming other methods by at least 11.2% in Rank-1 and 14.3% in mAP.Using only shared-visible parts during matching contributes an additional 3.3% Rank-1 and 0.9% mAP improvement.
- Parsing and ablations: Visualizations show that ISP identifies belongings and visible parts, while cascaded clustering produces finer pseudo-labels and avoids coarse or activation-driven assignments.The pseudo-labels evolve from coarse to fine, and performance is robust to different K values; K =5 is consistently worst in the reported ablation.
5 Conclusion
The paper proposes identity-guided human semantic parsing for aligned person re-identification using only person identity supervision. Experiments validate the method’s superiority.
- Identity-guided human semantic parsing locates human body parts and personal belongings at pixel-level using image-level person identity supervision.
- The approach targets aligned person re-identification without requiring extra semantic supervision.
- Extensive experiments validate the superiority of the proposed method.