PaperlayerView exact revision on arXiv

Source-linked AI summary

Person Re-identification by Contour Sketch under Moderate Clothing Change

Qize Yang, Ancong Wu, Wei-Shi Zheng

arXiv:2002.02295v1cs.CV

TL;DR

Existing person re-identification models rely on color appearance and assume clothing remains unchanged across camera views, creating a challenge when people change clothes. This paper uses contour sketches with a deep contour-sketch network to address moderate clothing changes, and reports higher accuracy than conventional methods while exposing remaining limitations for substantial changes.

  • Problem

    Existing state-of-the-art person re-identification models assume people do not change clothes across camera views, while unreliable clothing and face information challenges surveillance matching.

  • Method

    The paper performs cross-clothes re-identification using contour sketches and a deep network with spatial polar transformation and angle-specific feature extraction.

  • Results

    The proposed contour-sketch-based method achieves the highest accuracy for person re-identification under clothing change and outperforms conventional color-based methods.

  • Takeaways & Limitations

    Contour sketches provide effective visual cues for moderate clothing changes, while experiments show that cross-clothes person re-identification remains challenging.

  • Takeaways & Limitations

    The method is designed for moderate clothing change and remains limited when clothing changes substantially enough to cause large variation in body shape.

Abstract

from arXiv · show

Person re-identification (re-id), the process of matching pedestrian images across different camera views, is an important task in visual surveillance. Substantial development of re-id has recently been observed, and the majority of existing models are largely dependent on color appearance and assume that pedestrians do not change their clothes across camera views. This limitation, however, can be an issue for re-id when tracking a person at different places and at different time if that person (e.g., a criminal suspect) changes his/her clothes, causing most existing methods to fail, since they are heavily relying on color appearance and thus they are inclined to match a person to another person wearing similar clothes. In this work, we call the person re-id under clothing change the "cross-clothes person re-id". In particular, we consider the case when a person only changes his clothes moderately as a first attempt at solving this problem based on visible light images; that is we assume that a person wears clothes of a similar thickness, and thus the shape of a person would not change significantly when the weather does not change substantially within a short period of time. We perform cross-clothes person re-id based on a contour sketch of person image to take advantage of the shape of the human body instead of color information for extracting features that are robust to moderate clothing change. Due to the lack of a large-scale dataset for cross-clothes person re-id, we contribute a new dataset that consists of 33698 images from 221 identities. Our experiments illustrate the challenges of cross-clothes person re-id and demonstrate the effectiveness of our proposed method.

1 INTRODUCTION

The paper addresses cross-clothes person re-identification, where clothing changes make color-based matching unreliable. It proposes contour-sketch features and a new dataset, with experiments showing the method performs best under clothing change.

  • Existing re-id models mainly assume people retain the same clothes across camera views, leaving clothing-change matching insufficiently addressed.
  • Cross-clothes re-id is difficult because clothing changes can make color information unreliable or misleading and sharply reduce conventional model performance.
  • The method uses contour sketches instead of color cues, assuming moderate clothing changes preserve similar body shape.
  • The PRCC dataset contains 33698 images from 221 people, and experiments find the proposed method achieves the highest accuracy under clothing change.
  • SPT selects relatively invariant, discriminative contour patterns, while ASE extracts angle-specific fine-grained features in a deep contour-sketch network.

2 RELATED WORKS

Prior re-id methods rely substantially on color and texture, which become unreliable under clothing change. The paper instead derives contour sketches from RGB images and learns shape-based features without requiring depth sensors or painted sketches.

  • Hand-crafted descriptors, metric learning, and deep re-id models have been developed for cross-view variations including pose, scale, clutter, and occlusion.
  • These approaches rely substantially on color information, while contour and shape remain more consistent under moderate clothing change.
  • Depth-based methods capture 3D human shape but require an additional device that is not widely deployed.
  • The proposed method generates contour sketches from RGB images, using estimated 2D human shape rather than raw RGB or depth input.
  • Unlike sketch-retrieval work, this study matches contour sketches across camera views for clothing-change re-id; unlike gait recognition, it operates on still images without a complete gait cycle.

3.1 Problem Statement and Challenges

The problem arises because clothing and face information may be unreliable, while body shape can remain useful under moderate clothing changes. The main technical challenge is extracting discriminative local contour patterns because human contours can look globally similar.

  • Clothing changes can occur even in the short term, while face and clothing information may be unreliable for surveillance re-identification.
  • RGB histograms change substantially when the same person wears different clothes and can resemble those of another person in similar clothes.
  • Contour sketches are more consistent than RGB images when clothing changes are moderate, motivating shape-invariant feature learning.
  • Different people’s contour sketches can look globally similar, and many local curve patterns are not discriminative.
  • SPT selects discriminative curve patterns, ASE extracts robust fine-grained features, and a multistream framework aggregates multiple feature granularities.

3.2 Learning Spatial Polar Transformation (SPT) in Deep Neural Network

The method transforms contour sketches into polar coordinates and learns discriminative angular sampling rather than using fixed uniform sampling. This creates angle-organized representations and sector- or cone-shaped receptive fields for contour feature extraction.

  • Learnable spatial polar transformation: The proposed SPT transforms contour sketches into polar coordinates to select discriminative curve patterns and enhance rotation and scale invariance.The transformation uses polar angle to guide pattern selection.
  • Differentiable polar transformation: The differentiable polar transformation constructs a sampling grid from sampled angles and radii, then generates transformed pixels using interpolation.Sampled angles range from π to −π, while r_j = j × R/M defines sampled radii.
  • Learnable spatial polar transformation: SPT learns sampled angles so selective rather than uniformly distributed contour regions receive greater attention during transformation.The sampled angles are updated through learnable parameters while preserving their range and semantic order.
  • Learning sampled angles: The learned angle parameters preserve the sampled-angle range and ordering by mapping intermediate variables through a constrained linear function.This prevents direct optimization from disrupting the semantic order of angles.
  • Multistream representation: A multistream CNN uses different SPT layers and sampling ranges to obtain multiple transformed contour representations.The model combines coarse and local views, while each stream organizes transformed features into angle stripes and uses angle-specific extraction.
  • Transformation insight: After SPT, convolutional receptive fields become sector-shaped or cone-shaped relative to the original contour sketch instead of rectangular.Each transformed-image row corresponds to pixels sharing a polar angle.

3.3 Angle-Specific Extractor

The angle-specific extractor divides transformed feature maps into horizontal angle stripes and models each stripe separately. Unshared channel-dependency modules emphasize relatively invariant curve patterns while reducing local-noise effects.

  • Angle-specific feature extraction: The feature map is divided into B horizontal angle stripes, each average-pooled before angle-specific CNN branches refine the features.The stripes correspond to angle ranges in the transformed images.
  • Angle-specific extractor: Unshared ASE layers model channel interdependencies separately for different angle stripes because their dependencies differ.This gives greater attention to relatively invariant curve patterns.
  • Angle-specific extractor: Each ASE branch uses dimensionality reduction, ReLU activation, dimensionality incrementation, and sigmoid gating to compute channel dependencies.The input is the pooled feature vector for an angle stripe.
  • Noise reduction: A shortcut connection with element-wise summation reduces the influence of local noise in the extracted channel weights.The connection is introduced because channel weights from individual angle stripes may be corrupted by local noise.

3.4 Learning Multi-granularity Features

The multistream framework varies SPT sampling ranges to extract global coarse-grained and local fine-grained contour features. Different streams process different transformed regions and aggregate their representations.

  • Multi-granularity features: Varying the SPT sampling range enables the network to extract both coarse-grained global features and fine-grained local features.Different linear functions map intermediate variables to different angle ranges.
  • Multi-granularity features: Different streams receive different transformed images and extract features at multiple granularities.The resulting representations combine global and local contour information.

3.5 Learning Towards Clothing Invariant Features

The training objective treats differently clothed images of the same person as one identity and combines classification with triplet learning. This explicitly reduces cross-clothes intraclass differences while learning shared features for positive pairs.

  • Clothing-invariant supervision: Images of the same person wearing different clothes are assigned the same identity during training.The method seeks robust clothing-invariant curve patterns across their contour sketches.
  • Identity classification: Each branch and stream maps features to a C-dimensional identity distribution using a softmax classifier.The predicted distribution is compared with one-hot ground-truth labels using cross-entropy loss.
  • Triplet supervision: The auxiliary triplet loss uses anchor, positive, and negative concatenated features to reduce the intraclass gap and learn shared positive-pair representations.The positive sample has the same identity with different clothes.
  • Combined objective: The final training loss combines cross-entropy losses from the branches and streams with a weighted triplet margin loss.The triplet margin is m, and η weights the triplet term.

3.6 Summary of Our Model and Network Structure

The model transforms contour sketches with SPT layers to emphasize invariant, discriminative curve patterns, then uses CNN streams to learn multi-granularity features.

  • SPT transforms contour sketches to focus attention on relatively invariant and discriminative curve patterns.
  • A series of SPT layers uses different linear functions f to constrain the range of θ and produce varied transformed images.
  • The transformed images support CNN learning of multi-granularity features across streams.CNN extracts feature maps after the SPT layers for each stream.

4 A CROSS-CLOTHES RE-ID DATASET AND PRO-

Existing re-id datasets generally do not test clothing change, while PRCC provides a larger moderate-clothing-change benchmark with multiple camera views and additional variations.

  • Existing person re-id datasets are unsuitable for clothing-change evaluation because people wear the same clothes across different camera views.
  • Figure 5 pairs RGB images with corresponding contour sketches by person and groups other dataset variations by identity.
  • PRCC contains 221 identities across three camera views, with clothing changes in Camera C and same clothing across Cameras A and B.Cameras A and B capture different rooms, while Camera C uses a different day and different clothing.
  • PRCC includes clothing, illumination, occlusion, pose, and viewpoint variations, totaling 33698 images.Approximately 152 images are included for each person.
  • Contour sketches are generated with a holistically nested edge detection model using its fused output.
  • The dataset split uses 150 identities for training and 71 for testing, with no identity overlap; 25 percent of training images form validation data.

5 EXPERIMENTS

Experiments on PRCC evaluate implementation choices and compare contour-sketch and RGB-based person re-identification under same-clothes and cross-clothes conditions. The proposed method achieves the strongest reported cross-clothes accuracy, while ablations and visualizations indicate that its transformations and contour-based representation address clothing variation but remain limited by dramatic changes.

  • 5.2 Results on the PRCC Dataset: The proposed method achieves the best rank-1 accuracy, 34.38%, among compared methods for moderate clothing-change re-identification.It outperforms handcrafted features, deep-learning methods, and PCB on the cross-clothes task.
  • 5.2 Results on the PRCC Dataset: PCB (RGB) reaches 22.86% rank-1 accuracy on PRCC cross-clothes matching versus 92.3% on clothing-consistent Market-1501.This contrast illustrates the difficulty of re-identification when clothing changes.
  • 5.2 Results on the PRCC Dataset: RGB-based methods perform well without clothing changes but show a sharp performance drop when people dress differently.Attention and fine-grained feature learning remain potentially useful for clothing-invariant representation.
  • 5.2 Results on the PRCC Dataset: The contour-sketch method identifies targets across clothing changes in ranking visualizations, whereas PCB (RGB) tends to match people wearing similar clothes.A failure occurs when clothing changes dramatically, such as switching from leggings to slacks, adding a coat, and changing shoes.
  • 5.7 Ablation Study of the Proposed Model: Fixing the SPT angle or removing SPT lowers cross-clothes rank-1 accuracy to 31.05% or 25.74%, while removing triplet loss lowers it to 31.39%.These ablations support the contributions of learned spatial transformation and auxiliary clothing-invariant feature learning.
  • 5.7 Ablation Study of the Proposed Model: Removing SPT, ASE, and triplet loss reduces cross-clothes rank-1 accuracy to 21.95%, nearly 15% below the full model.Direct CNN convolution on original contour sketches is ineffective for extracting reliable and discriminative cross-clothes features.

5.8 Further Investigation of Our Model.

Further experiments analyze how SPT samples angles and transformation origins, showing trade-offs between discriminative coverage, detail preservation, computation, and feature consistency. Cross-dataset evaluations further examine performance on BIWI and other re-identification datasets.

  • Sampled angles: Increasing the number of sampled angles improves model performance but also increases computation, creating a trade-off.Table 8 evaluates both the number of sampled angles and strategies for learning them.
  • Angle ranges: SPT samples most frequently from contour ranges B, C, F, and G, indicating that these regions are more discriminative than ranges A and D.The comparison is based on the learned angle histograms before and after SPT training.
  • Angle ranges: Performance drops when the contour image is divided into more partitions, while top and bottom regions outperform left and right regions in one example strategy.The reported pattern holds for both cross-clothes matching and matching without clothing change.
  • Transformation origin: Using the contour-sketch center as SPT’s transformation origin performs similarly to alternatives without extra computation, making it practical.The study also compares image-center and pose-key-point origins, including the hip and neck.
  • Transformation origin: Replacing SPT with PTN degrades performance because PTN can learn different transformation origins for images of the same person, producing clearly different transformed images.Figure 13 contrasts PTN and SPT transformations for contour sketches from the PRCC dataset.
  • BIWI and cross-dataset evaluation: On BIWI, the proposed method outperforms hand-crafted features and PCB under both still and walking cross-clothes settings.The BIWI evaluation uses separate identity-disjoint training and testing splits, with still and walking comparisons against the training subset.

6 CONCLUSION

The paper addresses cross-clothes person re-identification by extracting features from contour sketches under moderate clothing changes. Its SPT- and ASE-based multistream framework is effective on the developed and BIWI datasets, but the task becomes intractable when clothing change combines with other large variations.

  • The paper extracts discriminative features from contour sketches to address moderate clothing changes in visible-light person re-identification.
  • The method combines learnable spatial polar transformation, angle-specific feature mining, and multistream aggregation of coarse- and fine-grained features.
  • Experiments on the developed dataset and BIWI validate the effectiveness and stability of the contour-sketch method compared with RGB-based methods.
  • The approach is limited to moderate clothing changes, while cross-clothes re-identification becomes intractable when clothing change is combined with other large variations.
Loading 2002.02295v1…