PaperlayerView exact revision on arXiv

Source-linked AI summary

Human Semantic Parsing for Person Re-identification

Mahdi M. Kalayeh, Emrah Basaran, Muhittin Gokmen, Mustafa E. Kamasak, Mubarak Shah

arXiv:1804.00216v1cs.CV

TL;DR

Person re-identification must distinguish identities despite background, pose, illumination, occlusion, and viewpoint variation, while existing local-feature extraction commonly uses bounding boxes. The paper proposes SPReID, which integrates human semantic parsing to extract local cues, and also evaluates simple full-image training with standard CNNs. These methods improve reported state-of-the-art performance across Market-1501, CUHK03, and DukeMTMC-reID.

  • Problem

    Person re-identification needs robust, discriminative representations despite substantial visual variation, while local body-part features commonly rely on bounding-box detection.

  • Method

    SPReID integrates human semantic parsing with convolutional re-identification features to use pixel-level body-region cues, alongside simple full-image CNN training.

  • Results

    The proposed methods improve reported state-of-the-art performance on Market-1501, CUHK03, and DukeMTMC-reID across mAP and rank-1 metrics.

  • Takeaways & Limitations

    Human semantic parsing can improve a strong person re-identification baseline, while properly trained standard CNN architectures can achieve state-of-the-art performance.

Abstract

from arXiv · show

Person re-identification is a challenging task mainly due to factors such as background clutter, pose, illumination and camera point of view variations. These elements hinder the process of extracting robust and discriminative representations, hence preventing different identities from being successfully distinguished. To improve the representation learning, usually, local features from human body parts are extracted. However, the common practice for such a process has been based on bounding box part detection. In this paper, we propose to adopt human semantic parsing which, due to its pixel-level accuracy and capability of modeling arbitrary contours, is naturally a better alternative. Our proposed SPReID integrates human semantic parsing in person re-identification and not only considerably outperforms its counter baseline, but achieves state-of-the-art performance. We also show that by employing a \textit{simple} yet effective training strategy, standard popular deep convolutional architectures such as Inception-V3 and ResNet-152, with no modification, while operating solely on full image, can dramatically outperform current state-of-the-art. Our proposed methods improve state-of-the-art person re-identification on: Market-1501 by ~17% in mAP and ~6% in rank-1, CUHK03 by ~4% in rank-1 and DukeMTMC-reID by ~24% in mAP and ~10% in rank-1.

1. Introduction

Person re-identification must learn identity-specific, context-invariant representations despite substantial variation across cameras, time, pose, illumination, occlusion, and image quality. The paper addresses this challenge with simple full-image models and human semantic parsing, reporting improvements across three benchmarks.

  • Person re-identification retrieves gallery images matching a query identity across cameras, making it a cross-camera data association problem.
  • Variation in illumination, clutter, occlusion, posture, observable body parts, and resolution creates large intra-class differences that hinder identity discrimination.
  • Part-level features can improve robustness over global representations, but bounding-box detection errors may propagate through the re-identification system.
  • A straightforward Inception-V3 model using full-body images and softmax cross-entropy at two resolutions can outperform current state-of-the-art.Re-ranking further increases the improvement margin.
  • Human semantic parsing replaces coarse body-part bounding boxes with pixel-level regions that model arbitrary contours under pose variation.The paper analyzes two integration variations and reports complementary representations.
  • ~17% in mAP on Market-1501, ~4% in rank-1 on CUHK03, and ~24% in mAP on DukeMTMC-reID improve reported state-of-the-art performance.The same reported results include ~6% and ~10% rank-1 gains on Market-1501 and DukeMTMC-reID, respectively.

2. Related Work

Prior person re-identification work improves representations with local body-part cues, attributes, and specialized losses, but commonly relies on complex detection or multi-stage pipelines. This paper introduces human semantic parsing as a pixel-accurate alternative to bounding-box part localization.

  • Deep re-identification research has focused on combining global representations with local body-part features to improve robustness.
  • Existing approaches use spatial transformers, region proposal networks, multi-stream CNNs, and fused global-local representations to model body parts.
  • Person attributes provide high-level semantic information but remain difficult to detect reliably under pose, illumination, and viewpoint variation.
  • Contrastive, binary, and triplet losses train re-identification models using positive-negative pairs or anchor-positive-negative tuples.
  • Human semantic parsing is proposed as a pixel-accurate alternative to bounding-box localization for extracting local body regions.The paper identifies this as its first integration of human semantic parsing into person re-identification.

3. Methodology

The methodology combines convolutional features with human semantic parsing to aggregate global and body-region cues for person re-identification. It uses Inception-V3 backbones, segmentation-specific modifications, and probability-map-based pooling within SPReID.

  • 3.1. Inception-V3 Architecture: Inception-V3 is used as the CNN backbone for both semantic parsing and person re-identification models.It is a 48-layer architecture that can operate on arbitrary image sizes through global average pooling.
  • 3.1. Inception-V3 Architecture: The original Inception-V3 reduces activation resolution to 1/8 within its first seven layers and produces a 2048-D vector after global average pooling.
  • 3.2. Human Semantic Parsing Model: The parsing model changes output stride from 32 to 16, replaces selected filters with dilated convolution, and adds atrous spatial pyramid pooling with rates 3,6,9,12.Global average pooling is removed and a 1×1 convolution performs pixel-level multi-class classification.
  • 3.3. Person Re-identification Model: SPReID contains a convolutional backbone, a semantic parsing branch, and two aggregation heads; its re-identification backbone removes global average pooling and outputs 2048 channels at stride 32.
  • 3.3. Person Re-identification Model: The framework transforms an RGB image into convolutional activations while generating semantic-region probability maps, then uses those maps to aggregate regional activations.
  • 3.3. Person Re-identification Model: SPReID pools backbone activations using normalized probability maps for foreground, head, upper-body, lower-body, and shoes.The maps provide local-region representations alongside the baseline global representation, with minimal added computation.

4. Experiments

The experiments evaluate SPReID and baseline person re-identification models across three benchmarks, varying datasets, input resolution, backbone architecture, and semantic parsing. Higher-resolution training and semantic parsing improve performance, while Inception-V3 offers competitive accuracy at lower computational cost than ResNet-152.

  • Evaluation setup: The evaluation uses Market-1501, CUHK03, and DukeMTMC-reID, with additional datasets aggregated into a training set of approximately 111,000 images.The training data contains approximately 17,000 identities from 10 person re-identification datasets.
  • Evaluation setup: The human semantic parsing model is trained on LIP and groups 20 semantic labels into five coarse body-region labels for person re-identification.The five coarse labels are foreground, head, upper-body, lower-body, and shoes.
  • Baseline analysis: Higher-resolution input improves mAP and re-identification rate across all three datasets, with smaller gaps for rank-10 than rank-1.Fine-tuning Model-L at 748×246 adds an average of 4.75% mAP and 1.71% rank-1 over Model-L.
  • Baseline analysis: Inception-V3 provides competitive performance with ResNet-152 and significantly outperforms ResNet-50 in the baseline comparison.ResNet-152 is three times more computationally expensive than Inception-V3, motivating the choice of Inception-V3 as the main backbone.
  • SPReID analysis: SPReID improves the Inception-V3 baseline on Market-1501, CUHK03, and DukeMTMC-reID by exploiting human semantic parsing for local feature aggregation.The reported gains are 6.61% mAP and 2.58% rank-1 on Market-1501, 3.33% rank-1 on CUHK03, and 8.91% mAP and 4.22% rank-1 on DukeMTMC-reID.
  • SPReID analysis: The combined SPReID representation performs better than the Inception-V3 baseline, although the gap decreases when compared with the stronger ResNet-152 baseline.Re-ranking further improves the reported performance.

5. Implementation Details

Implementation details specify optimization settings for person re-identification and semantic parsing, including learning-rate schedules, gradient controls, and semantic-parsing training resolution.

  • Person re-identification: Person re-identification uses minibatches of 15, momentum 0.9, weight decay 0.0005, and gradient clipping at 2.0.The initial learning rate is 0.01 in the first phase and 0.001 in the second.
  • Person re-identification: The re-identification learning rate is decayed ten times using exponential shift with rate 0.9, with Nesterov Accelerated Gradient and ImageNet-pretrained initialization.
  • Human semantic parsing: The human semantic parsing model is trained for 30K iterations using Inception-V3, atrous spatial pyramid pooling, and a 1×1 convolution layer.It uses initial learning rates of 0.01, 0.1, and 0.1 respectively, with 512×512 input images.

6. Conclusion

The paper addresses whether person re-identification requires complex models or bounding-box body parts, showing that simple training and human semantic parsing answer both questions positively.

  • Simple deep convolutional architectures trained properly on many high-resolution images can outperform current state-of-the-art.
  • Human semantic parsing further improves a state-of-the-art person re-identification baseline by providing a more natural way to use human body parts.
  • SPReID applies minimal modifications to the person re-identification backbone while exploiting human semantic parsing.
  • The experiments support investing in human semantic parsing for person re-identification.
Loading 1804.00216v1…