PaperlayerView exact revision on arXiv

Source-linked AI summary

PersonNet: Person Re-identification with Deep Convolutional Neural Networks

Lin Wu, Chunhua Shen, Anton van den Hengel

arXiv:1601.07255v2cs.CV

TL;DR

Person re-identification requires matching pedestrians across camera views despite complex visual variation, while conventional approaches may separate feature extraction from metric learning. PersonNet jointly learns features and similarity from image pairs using neighborhood differences, deep 3×3-filter CNNs, and RMSProp. It reports state-of-the-art performance across CUHK03, Market-1501, and CUHK01, including 71.14% rank-1 recognition on CUHK01 versus 65% for the next best method.

  • Problem

    Person re-identification must estimate whether pedestrian-image pairs depict the same person despite varied camera views and complex visual changes, while many metric-learning methods use two-stage pipelines.

  • Method

    PersonNet is an end-to-end deep network that jointly learns features and similarity from paired RGB images using neighborhood patch differences, 7–10 weight layers with 3×3 filters, and RMSProp.

  • Results

    PersonNet consistently outperforms state-of-the-art across CUHK03, Market-1501, and CUHK01; on CUHK01 it achieves 71.14% rank-1 recognition versus 65% for the next best method.

  • Takeaways & Limitations

    The experiments support increased representation depth as beneficial for person identity matching and show state-of-the-art performance on the evaluated datasets.

Abstract

from arXiv · show

In this paper, we propose a deep end-to-end neu- ral network to simultaneously learn high-level features and a corresponding similarity metric for person re-identification. The network takes a pair of raw RGB images as input, and outputs a similarity value indicating whether the two input images depict the same person. A layer of computing neighborhood range differences across two input images is employed to capture local relationship between patches. This operation is to seek a robust feature from input images. By increasing the depth to 10 weight layers and using very small (3$\times$3) convolution filters, our architecture achieves a remarkable improvement on the prior-art configurations. Meanwhile, an adaptive Root- Mean-Square (RMSProp) gradient decent algorithm is integrated into our architecture, which is beneficial to deep nets. Our method consistently outperforms state-of-the-art on two large datasets (CUHK03 and Market-1501), and a medium-sized data set (CUHK01).

I. INTRODUCTION

Person re-identification matches pedestrian images across nonoverlapping camera views despite substantial visual variation. PersonNet addresses this by jointly learning representations and similarity through a deep CNN with neighborhood matching and RMSProp, achieving strong benchmark results.

  • Person re-identification must match pedestrians across nonoverlapping cameras despite changes in appearance, pose, illumination, clutter, occlusion, resolution, and camera placement.
  • Traditional metric-learning pipelines often extract hand-crafted features before learning a metric, which can produce sub-optimal solutions.
  • CNNs enable joint feature representation and metric learning, while prior person re-identification networks introduced patch matching and cross-input neighborhood differences.
  • PersonNet uses ten weight layers with 3×3 convolution filters and online pair sampling, integrating RMSProp to handle widely varying gradients in deep networks.
  • Benchmark experiments report the best results on three popular person re-identification datasets.

II. RELATED WORK

Prior person re-identification work developed hand-crafted representations, supervised metric learning, and early deep CNN architectures. PersonNet differs through greater depth, small filters, and RMSProp, and reports superior performance across multiple datasets.

  • Earlier representation methods describe pedestrians using global color histograms, local patterns, symmetry, and weighted color features.
  • Supervised metric-learning methods seek projections that keep same-person feature vectors close and different-person vectors far apart.
  • Deep re-identification systems included Siamese CNNs with cosine distance, patch matching, and cross-input neighborhood difference features.
  • PersonNet uses 7–10 weight layers, 3×3 filters, and RMSProp, distinguishing its architecture from previous networks.

III. THE ARCHITECTURE

PersonNet processes paired fixed-size RGB pedestrian images through tied convolutional layers, pooling, and fully connected layers to classify whether the pair depicts the same person.

  • PersonNet takes a pair of fixed-size 160×60 RGB images as input.
  • The paired images pass through convolutional layers with 3×3 filters and three max-pooling layers using 2×2 windows with stride 2.
  • After convolution, three fully connected layers have dimensions 4096, 4096, and 512.
  • A softmax layer determines whether the two input images show the same or different person.

A. Convolution and max pooling

The convolution and pooling components extract shared local features from paired pedestrian images while progressively reducing spatial variation and dimensionality.

  • Shared filters convolve the two camera-view images to produce local feature responses.
  • PersonNet uses 3×3 filters throughout the network with stride 1 instead of larger filters in the first convolutional layers.
  • Stacks of two 3×3 convolutional layers provide a 5×5 receptive field while introducing more nonlinear activation functions.
  • The network uses hyperbolic tangent activations to rescale linear outputs to the range [-1, 1].
  • Max-pooling reduces output dimensionality and variation from slight translations in deformable objects.

B. Modeling neighborhood patch matching

PersonNet compares local neighborhoods of feature responses across two pedestrian views, using these differences to model displacement from pose and viewpoint variation.

  • B. Modeling neighborhood patch matching: The patch matching layer computes differences between local filter-response neighborhoods across two input views.Each feature location produces a neighborhood-difference representation rather than only comparing corresponding responses.
  • B. Modeling neighborhood patch matching: For each of 32×7 feature locations, the layer produces a 3×3 block of differences, yielding feature maps K_i ∈ R32×7×3×3.
  • B. Modeling neighborhood patch matching: The 3×3 neighborhood models body-part displacement caused by pose and viewpoint variations.
  • B. Modeling neighborhood patch matching: PersonNet places the matching layer between convolutional processing and higher layers that compute relationships between the two images.

IV. TRAINING STRATEGIES

PersonNet uses RMSProp rather than SGD to update parameters, adapting gradient normalization across weights and mini-batches for more robust learning.

  • IV. TRAINING STRATEGIES: RMSProp updates parameters by dividing each gradient using a running average of its recent magnitude.
  • IV. TRAINING STRATEGIES: Unlike SGD’s shared learning-rate behavior, RMSProp uses a different divisor for each mini-batch while keeping adjacent divisors similar.
  • IV. TRAINING STRATEGIES: The method maintains a moving average of the squared gradient for each weight.
  • IV. TRAINING STRATEGIES: The authors report that RMSProp makes learning work better and performs more robustly than SGD in their architecture.

B. Data augmentation and data balancing

The training strategy addresses pair imbalance through translation-based augmentation, online balancing of positive and negative pairs, and regularization during optimization.

  • B. Data augmentation and data balancing: Positive training pairs are several orders fewer than negative pairs, creating data imbalance and overfitting concerns.
  • B. Data augmentation and data balancing: The dataset is augmented by randomly sampling five translated images around each pedestrian image center.
  • B. Data augmentation and data balancing: PersonNet online-samples equal numbers of negative and positive pairs instead of gradually increasing negatives to a fixed 5:1 ratio.
  • B. Data augmentation and data balancing: Training uses online pair sampling with RMSProp, L2 and dropout regularization, and a mini-batch size of 2.
  • B. Data augmentation and data balancing: Learning is stopped within 100K iterations, with the rate reduced tenfold when validation accuracy stops improving.

A. Experimental settings

Experiments evaluate PersonNet on three benchmark datasets using repeated single-shot CMC measurements and comparisons with established state-of-the-art methods.

  • A. Experimental settings: Experiments use the CUHK03, CUHK01, and Market-1501 person re-identification benchmarks.
  • A. Experimental settings: Under the single-shot protocol, each probe is matched against the gallery and the true-match rank is recorded.
  • A. Experimental settings: The rank-k recognition rate and cumulative rank values are reported as CMC results averaged over ten evaluations.
  • A. Experimental settings: PersonNet is compared with state-of-the-art metric-learning, feature-based, ensemble, and deep re-identification approaches.

B. Experiments on CUHK03 data set

Experiments on CUHK03 use labeled pedestrian bounding boxes and compare Rank-1, Rank-5, and Rank-10 recognition rates across methods. PersonNet improves Rank-1 from 62.1% for Ensemble to 64.8%.

  • The CUHK03 dataset contains 13,164 images from 1,360 pedestrians observed across six surveillance cameras.
  • The experiments use manually labeled pedestrian bounding boxes rather than automatically detected boxes.
  • Results are reported using Rank-1, Rank-5, and Rank-10 recognition rates for various methods.
  • 64.8% Rank-1 recognition rate improves on Ensemble’s 62.1% result.

C. Experiments on CUHK01 data set

Experiments on CUHK01 evaluate PersonNet with 100 identities for testing and 871 for training. PersonNet achieves a 71.14% Rank-1 recognition rate, exceeding the next-best method’s 65%.

  • CUHK01 contains 971 identities with two images per person in each view.
  • The evaluation follows a split with 100 identities for testing and 871 identities for training.
  • 71.14% Rank-1 recognition rate exceeds the next-best method’s 65%.
  • The reported comparison is against previous methods and describes PersonNet as superior by a large margin.

E. Convergence study

The convergence study compares SGD and RMSProp empirically. RMSProp is reported as more stable and relatively faster to converge than SGD, while the conclusion links deep networks to recognition accuracy.

  • The study empirically examines convergence speed for SGD and RMSProp.
  • RMSProp is reported as more stable and relatively faster to converge than SGD.
  • SGD can have unstable update steps because it depends on the current batch of instances.
  • RMSProp keeps a running average of recent gradient magnitudes and divides the next gradient by this average.
  • The conclusion reports that representation depth benefits recognition accuracy in person matching.
Loading 1601.07255v2…