PaperlayerView exact revision on arXiv

Source-linked AI summary

Harmonious Attention Network for Person Re-Identification

Wei Li, Xiatian Zhu, Shaogang Gong

arXiv:1802.08122v1cs.CV

TL;DR

Person re-identification remains difficult when automatically detected images are misaligned by clutter, occlusion, missing parts, and pose variation. The paper proposes HA-CNN, a lightweight end-to-end model that jointly learns soft pixel and hard regional attention with feature representations. HA-CNN outperforms state-of-the-art methods across CUHK03, Market-1501, and DukeMTMC-ReID, while analyses support complementary benefits from its attention components.

  • Problem

    Automatically detected person images are often misaligned by background clutter, occlusion, missing body parts, and pose variation, limiting re-id matching across camera views.

  • Method

    HA-CNN jointly learns soft spatial-channel and hard regional attention with re-id feature representations in a lightweight end-to-end CNN, including cross-attention interaction learning.

  • Results

    HA-CNN outperforms a wide range of state-of-the-art re-id models on CUHK03, Market-1501, and DukeMTMC-ReID.

  • Takeaways & Limitations

    Jointly modelling complementary soft and hard attention can improve person re-id under arbitrary bounding-box alignment while retaining a lightweight architecture.

Abstract

from arXiv · show

Existing person re-identification (re-id) methods either assume the availability of well-aligned person bounding box images as model input or rely on constrained attention selection mechanisms to calibrate misaligned images. They are therefore sub-optimal for re-id matching in arbitrarily aligned person images potentially with large human pose variations and unconstrained auto-detection errors. In this work, we show the advantages of jointly learning attention selection and feature representation in a Convolutional Neural Network (CNN) by maximising the complementary information of different levels of visual attention subject to re-id discriminative learning constraints. Specifically, we formulate a novel Harmonious Attention CNN (HA-CNN) model for joint learning of soft pixel attention and hard regional attention along with simultaneous optimisation of feature representations, dedicated to optimise person re-id in uncontrolled (misaligned) images. Extensive comparative evaluations validate the superiority of this new HA-CNN model for person re-id over a wide variety of state-of-the-art methods on three large-scale benchmarks including CUHK03, Market-1501, and DukeMTMC-ReID.

1. Introduction

Person re-identification must match people across non-overlapping camera views despite misaligned, auto-detected bounding boxes and changing poses. HA-CNN addresses this by jointly learning complementary attention and feature representations in a lightweight end-to-end model.

  • 1. Introduction: Auto-detected person boxes contain background clutter, occlusion, and missing body parts, creating severe cross-view matching misalignment.People are also captured in varied poses across open space and time.
  • 1. Introduction: Existing attention methods use local patch calibration or saliency weighting, but handcrafted approaches do not jointly learn expressive features and matching metrics end-to-end.
  • 1. Introduction: HA-CNN jointly learns multi-granularity attention selection and feature representation in a more lightweight network architecture.
  • 1. Introduction: HA-CNN simultaneously learns hard region-level and soft pixel-level attention within arbitrary person boxes while optimising re-id feature representations.Its design maximises complementary information between attention selection and feature discrimination.
  • 1. Introduction: Cross-attention interaction learning further enhances compatibility between attention selection and feature representation under re-id discriminative constraints.Comparative evaluations cover CUHK03, Market-1501, and DukeMTMC-ReID.

2. Related Work

Prior re-id research includes alignment assumptions, handcrafted attention calibration, and deep models that generally focus on regional attention. HA-CNN instead models multiple correlated attention levels jointly within one re-id model.

  • 2. Related Work: Most identity-discriminative re-id methods assume well-aligned person images, an assumption challenged by imperfect detections and changing human poses.
  • 2. Related Work: Handcrafted local patch matching and saliency weighting were introduced to address matching misalignment.
  • 2. Related Work: Recent deep re-id methods incorporate regional attention sub-networks, including models using external pose supervision or latent hard-attention region localization.
  • 2. Related Work: HA-CNN jointly models soft and hard attention in a single re-id model and adds cross-attention interaction learning to exploit their complementary effect.The paper identifies this multi-level correlated attention formulation as distinct from existing single-level attention methods.
  • 2. Related Work: The proposed attention CNN architecture also targets deployment scalability, which is important for large surveillance video data.The paper connects this concern to sparse labels, overfitting risk, and slow feature extraction.

3. Harmonious Attention Network

HA-CNN jointly learns complementary soft and hard attention with global and local re-id representations in a lightweight multi-branch CNN. Its hierarchical design progressively refines attention while cross-attention interaction couples global and local learning under shared re-id constraints.

  • Network overview: HA-CNN concurrently learns global and local feature representations with complementary hard regional and soft spatial-channel attention from arbitrarily aligned bounding boxes.The local branch models person regions, while the global branch models the entire image.
  • Network overview: The architecture uses one global branch and T identical local streams to reduce model complexity while preserving network depth.The global branch processes the entire person image, and each local stream targets one of T local regions.
  • Harmonious attention learning: Hard regional attention searches coarse latent discriminative regions, complementing soft attention's fine-grained pixel selection within a block-wise hierarchical design.The module-wise arrangement progressively refines attention and reduces the attention search space while enriching final representations with multi-scale selectiveness.
  • Harmonious attention learning: Soft attention produces a saliency map matching the input feature dimensions by factorising spatial pixel attention and channel scale attention.Spatial attention is shared across channels, while channel attention is shared across spatial dimensions; tensor multiplication reconstructs the full map efficiently.
  • Cross-attention interaction: Cross-attention interaction transfers attended global information to local features and shares learning capability across branches under joint re-id optimisation.The global branch receives gradients from both global and local losses, reducing local-branch complexity through partial feature-learning sharing.
  • Cross-attention interaction: HA-CNN combines joint attention generation with joint attention application to complete end-to-end optimisation of attention selection for person re-id.The design shares most learning with the target-task network through a smaller multi-task sub-network and models soft and hard attention together.

4. Experiments

Experiments evaluate HA-CNN on three person re-identification benchmarks using standard protocols and CMC/mAP metrics. Results show strong accuracy, complementary attention benefits, effective local-global fusion, and lightweight model complexity.

  • Comparisons to State-of-the-Art Methods: HA-CNN outperforms the second-best JLML model on Market-1501 by 6.1% Rank-1 and 10.2% mAP in SQ.The corresponding HA-CNN and JLML values are 91.2 and 85.1 for Rank-1, and 75.7 and 65.5 for mAP.
  • Model Complexity: HA-CNN has 2.7 million parameters, compared with SVDNet’s 23.5 million, while requiring no data augmentation or ImageNet pre-training.The paper links the smaller model to lower training and test cost, easier training, and faster deployment.
  • Evaluation Protocol: HA-CNN is evaluated on Market-1501, DukeMTMC-ReID, and CUHK03, including manually labelled and auto-detected CUHK03 bounding boxes.The evaluation uses standard person re-identification training/test splits and protocols, with CMC and mAP as performance measures.
  • Attention Analysis: Combining soft spatial, soft channel, and hard regional attention produces further performance gains over individual attention components.The analysis attributes the gains to complementary information between fine-grained soft attention and coarse hard attention.
  • Attention Analysis: Cross-attention interaction learning improves Market-1501 Rank-1 by 4.6% and mAP by 9.5%, and DukeMTMC-ReID Rank-1 by 6.5% and mAP by 8.4%.The comparisons are between models with and without cross-attention interaction learning under SQ.
  • Feature Analysis: Joint local and global features improve Market-1501 performance by 6.1% Rank-1 and 10.2% mAP over the compared alternative.The results support complementary effects from jointly learning local and global representations.
  • Attention Visualisation: Visualisations show hard attention localising four body parts and soft attention progressively focusing on salient pixels across three levels.The levels move from whole-body spatial attention toward local salient parts.

5. Conclusion

The paper presents HA-CNN for end-to-end joint learning of attention selection and feature representations in person re-identification. Across three benchmarks, the lightweight architecture combines complementary attention mechanisms and interaction learning to address misaligned images.

  • Conclusion: HA-CNN jointly learns multiple complementary attention types and re-identification feature representations in a unified lightweight CNN.Its two-branch architecture and Harmonious Attention module support end-to-end learning.
  • Conclusion: The model targets matching misalignment in manually labelled and more challenging auto-detected person images.The paper contrasts this setting with methods that ignore misalignment or use stringent attention learning algorithms.
  • Conclusion: Extensive evaluations on three re-identification benchmarks validate HA-CNN against a wide range of state-of-the-art methods.The paper also analyses model components and compares model complexity with popular alternatives.
Loading 1802.08122v1…