PaperlayerView exact revision on arXiv

Source-linked AI summary

A Siamese Long Short-Term Memory Architecture for Human Re-Identification

Rahul Rama Varior, Bing Shuai, Jiwen Lu, Dong Xu, Gang Wang

arXiv:1607.08381v1cs.CV

TL;DR

Human re-identification must match pedestrians across cameras despite cluttered backgrounds and changes in appearance, illumination, and pose. This paper introduces a siamese LSTM that processes image regions sequentially and selectively propagates contextual information. It consistently outperforms a no-LSTM baseline and achieves promising results against state-of-the-art methods on several datasets.

  • Problem

    Human re-identification requires matching pedestrians across camera views despite cluttered backgrounds and variation in appearance, illumination, and pose, while existing feature representations are formed locally and independently.

  • Method

    The paper uses a siamese LSTM trained with contrastive loss to process image regions sequentially, learn spatial dependencies, and selectively propagate relevant contextual information.

  • Results

    The approach consistently outperforms the baseline and achieves promising results compared to state-of-the-art methods on Market-1501, CUHK03, and VIPeR.

  • Takeaways & Limitations

    LSTM gating can selectively propagate relevant context and enhance the discriminative capacity of local features for human re-identification.

Abstract

from arXiv · show

Matching pedestrians across multiple camera views known as human re-identification (re-identification) is a challenging problem in visual surveillance. In the existing works concentrating on feature extraction, representations are formed locally and independent of other regions. We present a novel siamese Long Short-Term Memory (LSTM) architecture that can process image regions sequentially and enhance the discriminative capability of local feature representation by leveraging contextual information. The feedback connections and internal gating mechanism of the LSTM cells enable our model to memorize the spatial dependencies and selectively propagate relevant contextual information through the network. We demonstrate improved performance compared to the baseline algorithm with no LSTM units and promising results compared to state-of-the-art methods on Market-1501, CUHK03 and VIPeR datasets. Visualization of the internal mechanism of LSTM cells shows meaningful patterns can be learned by our method.

1 Introduction

Human re-identification matches pedestrians across camera views despite substantial visual variation. The paper proposes a siamese LSTM that uses sequential regional context to improve local representations and reports stronger performance than a no-LSTM baseline and promising state-of-the-art comparisons.

  • Human re-identification retrieves gallery images matching a query pedestrian across different surveillance cameras, but clutter, illumination, pose, and appearance variation make matching difficult.
  • Existing feature extraction methods represent image regions locally and independently, motivating contextual modeling across regions.
  • The network uses a contrastive loss to pull similar image pairs together in embedding space and push dissimilar pairs apart.
  • The proposed siamese LSTM processes horizontal image regions sequentially and selectively retains or discards information using current and previous inputs.
  • The approach achieves better performance than a baseline without LSTM units and promising results against several state-of-the-art re-identification algorithms.
  • The authors describe the architecture as the first siamese architecture using LSTM as its fundamental component for human re-identification.

2 Related Works

Related work centers on handcrafted or learned feature representations, distance metrics, and CNN architectures for human re-identification. RNNs model spatial context, while LSTMs add selective information-flow control for longer or more relevant dependencies.

  • Human re-identification research commonly develops feature representations or learns distance metrics, including color, texture, and invariant local descriptors.
  • The proposed architecture is illustrated as processing image features sequentially, combining hidden representations through a learned mapping before contrastive comparison.
  • RNNs capture contextual dependencies by retaining information from previously processed inputs and are used across sequential and computer-vision tasks.
  • Traditional RNNs can struggle with long sequences because of vanishing gradients, motivating the introduction of LSTM networks.
  • Unlike traditional RNNs, LSTM cells use multiplicative gating interactions to selectively allow or block information flow through the network.

3 Our Framework

The framework uses a siamese LSTM to process horizontal image regions sequentially, preserving contextual dependencies while learning an embedding that brings similar pedestrian pairs together and separates dissimilar pairs. Shared LSTM branches transform region features before distance-based contrastive optimization.

  • 3 Our Framework: The model uses two parameter-sharing LSTM branches to match pedestrian images from different surveillance cameras.The branches process paired images and are optimized with contrastive loss.
  • 3 Our Framework: LSTM hidden representations incorporate current and previous region information, allowing contextual dependencies to influence local features.The recurrent state retains information from earlier regions, while gating controls memory updates and exposure to other network components.
  • 3 Our Framework: Each image is divided into horizontal stripes and treated as a spatial sequence for sequential LSTM processing.The row-wise representation supports translational invariance across viewpoints.
  • 3 Our Framework: LOMO and Color Names features are extracted from image rows and supplied as inputs to the parallel LSTM networks.LOMO uses multi-scale row representations, while Color Names features produce row-wise representations from block-level features.
  • 3 Our Framework: The hidden representations from all regions are combined into image representations whose distance is optimized with contrastive loss.The loss attracts similar pairs and separates dissimilar pairs by a margin; hard negatives are sampled during training.
  • 3 Our Framework: The siamese branches independently back-propagate feature gradients through time, then sum gradients for their shared parameters before updating the weights.Training uses mini-batch stochastic gradient descent with RMSProp-related optimization settings.

4 Experiments

The experiments evaluate the proposed method against LSTM-free single-layer and multi-layer baselines and compare performance across three human re-identification datasets. The LSTM architecture outperforms both baseline variants on all evaluated datasets, while contextual feature selection is reported as effective.

  • 4 Experiments: The evaluation reports both CMC and mAP on Market-1501, CUHK03, and VIPeR for comparison with baselines and state-of-the-art algorithms.The datasets include Market-1501, CUHK03, and VIPeR, with Market-1501 containing 32,668 annotated bounding boxes of 1,501 subjects.
  • 4 Experiments: The baseline removes LSTM units, concatenates features from all rows, and uses the same contrastive loss as the proposed method.A multi-layer baseline is formed by extending the single-layer baseline framework.
  • 4 Experiments: The proposed LSTM architecture outperforms single-layer and multi-layer baseline algorithms on all three datasets.
  • 4 Experiments: Contextual-dependency-based feature selection is reported to be effective for human re-identification tasks.On VIPeR, the 3-layer baseline has lower Rank 1 performance than the 2-layer and 1-layer approaches, which the authors attribute possibly to over-fitting on the smaller dataset.
  • 4 Experiments: Model settings are selected by cross-validation, with hidden dimensions of 25 for Market-1501 and 50 for CUHK03 and VIPeR.Validation performance showed no significant improvement above 25 on Market-1501 and declined beyond a margin setting.

5 Analysis

The analysis examines LSTM gate behavior and state-of-the-art comparisons across Market-1501, CUHK03, and VIPeR. Gate activations selectively propagate contextual information, while benchmark performance varies by dataset.

  • Internal Mechanisms of the LSTM cell: Figure 3 pairs two VIPeR query images and input-gate heat maps with retrieved gallery images, marking correct matches in green.The heat maps visualize gate responses but their L2 norms are not used directly by the system.
  • Internal Mechanisms of the LSTM cell: The input gate attenuates background and head regions while propagating relevant upper-body information across 24 image-region steps.Gate activations are represented by L2 norms of 50-dimensional vectors; darker red indicates propagation and deeper blue indicates blocking.
  • State-of-the-art Comparison: The tables compare the proposed method with state-of-the-art algorithms on Market-1501, CUHK03, and VIPeR.Market-1501 results are summarized in Table 2, CUHK03 in Table 3, and VIPeR individual-method results in Table 4.
  • State-of-the-art Comparison: On Market-1501, a recent metric-learning approach outperforms the proposed method, whereas the authors view metric learning as complementary to feature learning.The paper positions its main contribution in feature learning rather than metric learning.
  • State-of-the-art Comparison: On CUHK03, the proposed method achieves the best Rank 1 results among compared individual approaches, while several methods outperform it on VIPeR.The authors attribute the VIPeR gap to fewer positive pairs and fewer training identities, and suggest transfer learning or data augmentation.

6 Conclusion and Future Works

The paper concludes that its siamese LSTM architecture selectively propagates contextual information to strengthen local features. It reports consistent gains over the baseline and promising comparisons with state-of-the-art methods, while VIPeR remains a challenging setting.

  • Conclusion: The proposed siamese LSTM architecture learns spatial dependencies and selectively propagates or blocks contextual information through its gating mechanisms.The analysis examines input, forget, and output gate activation statistics to show how relevant information is retained.
  • Future Works: VIPeR comparisons use individual methods, and several recent approaches outperform the proposed approach on that dataset.The paper reports this comparison in its state-of-the-art evaluation.
  • Conclusion: The approach is evaluated on challenging real-world human re-identification datasets and consistently outperforms the baseline while achieving promising state-of-the-art comparisons.The conclusion summarizes the reported evaluation across several datasets.
Loading 1607.08381v1…