PaperlayerView exact revision on arXiv

Source-linked AI summary

Improving Pedestrian Attribute Recognition With Weakly-Supervised Multi-Scale Attribute-Specific Localization

Chufeng Tang, Lu Sheng, Zhaoxiang Zhang, Xiaolin Hu

arXiv:1910.04562v1cs.CV

TL;DR

Pedestrian attribute recognition must localize attribute-related regions without region annotations, but existing attention and body-part methods do not define local areas by attribute. The paper introduces multi-scale Attribute Localization Modules with feature-pyramid guidance and reports significant improvement across PETA, RAP, and PA-100K. Its analysis indicates that the method can localize informative regions weakly supervised.

  • Problem

    Regional localization is challenging because pedestrian attribute recognition lacks region annotations and existing methods do not use attributes to define local feature areas.

  • Method

    The framework uses Attribute Localization Modules with channel attention and spatial transformation across multiple feature levels, guided by a feature pyramid and trained end-to-end with deep supervision.

  • Results

    The method significantly outperforms most existing methods on PETA, RAP, and PA-100K, with 3.7% higher mA and 1.4% higher F1 than the second-best results on PA-100K.

  • Takeaways & Limitations

    The framework can discover informative attribute-specific regions without additional region annotations while improving pedestrian attribute recognition.

Abstract

from arXiv · show

Pedestrian attribute recognition has been an emerging research topic in the area of video surveillance. To predict the existence of a particular attribute, it is demanded to localize the regions related to the attribute. However, in this task, the region annotations are not available. How to carve out these attribute-related regions remains challenging. Existing methods applied attribute-agnostic visual attention or heuristic body-part localization mechanisms to enhance the local feature representations, while neglecting to employ attributes to define local feature areas. We propose a flexible Attribute Localization Module (ALM) to adaptively discover the most discriminative regions and learns the regional features for each attribute at multiple levels. Moreover, a feature pyramid architecture is also introduced to enhance the attribute-specific localization at low-levels with high-level semantic guidance. The proposed framework does not require additional region annotations and can be trained end-to-end with multi-level deep supervision. Extensive experiments show that the proposed method achieves state-of-the-art results on three pedestrian attribute datasets, including PETA, RAP, and PA-100K.

1. Introduction

Pedestrian attribute recognition benefits from regional features, but existing holistic, attribute-agnostic, and predefined-part methods do not reliably establish attribute-region correspondence. The proposed framework performs weakly supervised, attribute-specific localization at multiple scales and reports improvement on three datasets.

  • 1. Introduction: Regional features are important for fine-grained attribute classification, whereas holistic methods extract representations only from whole pedestrian images.
  • 1. Introduction: Attribute-agnostic attention captures broad regions without making them specific to the target attribute, such as Longhair.
  • 1. Introduction: Part-based methods use predefined or externally localized body parts, but their fused features still fail to indicate attribute-region correspondence and require extra computation.
  • 1. Introduction: The Attribute Localization Module combines channel attention with a spatial transformer to discover discriminative regions and learn attribute-specific regional representations.
  • 1. Introduction: The proposed framework performs weakly supervised, attribute-specific localization at multiple scales and achieves significant improvement on PETA, RAP, and PA-100K.It is end-to-end trainable and uses a feature pyramid that combines low-level details with high-level semantics.

2. Related Works

Prior pedestrian attribute recognition methods progressed from handcrafted and holistic CNN features toward weakly supervised localization, attention, and feature-pyramid designs. The paper combines these ideas into attribute-specific multi-level localization for pedestrian attributes.

  • 2. Related Works: CNN-based pedestrian attribute recognition improved over handcrafted features, while holistic models formulate recognition as multi-label classification from whole images.
  • 2. Related Works: Spatial Transformer Networks enable end-to-end attention localization without region annotations and have been applied to body-part and discriminative-region discovery.
  • 2. Related Works: The proposed framework uses bottom-up and top-down pathways, multiple Attribute Localization Modules, deep supervision, and element-wise maximum aggregation.
  • 2. Related Works: Each Attribute Localization Module combines channel attention with a simplified spatial transformer and serves one attribute at one feature level.
  • 2. Related Works: The feature pyramid architecture extends cross-level feature integration toward attentive-region localization for pedestrian attribute recognition.

3. Proposed Method

The framework combines a multi-level feature pyramid with attribute-specific localization modules to discover discriminative regions without region annotations. Deep supervision trains predictions from multiple branches, whose responses are fused during inference.

  • 3.1. Network Architecture: The feature pyramid combines low-level details with high-level semantics to support localization and regional feature learning at multiple scales.Bottom-up features come from three BN-Inception levels, while top-down connections transmit and upsample higher-level features before adjacent levels are concatenated.
  • 3.2. Attribute Localization Module: Attribute-specific localization uses ALMs to discover discriminative regions for each attribute without requiring region annotations.Each ALM uses channel recalibration and a spatial transformer to extract attribute-specific regional features.
  • 3.2. Attribute Localization Module: The simplified spatial transformer represents each attribute region as a differentiable bounding box and remains end-to-end trainable without region annotations.Scaling and translation parameters determine the bounding box, with constrained ranges applied through sigmoid and tanh activations.
  • 3.2. Attribute Localization Module: A channel-attention sub-network recalibrates combined features so different attributes can emphasize different proportions of fine details and strong semantics.Reweighted features retain complementary information through an additional residual link before transformation estimation and regional sampling.
  • 3.3. Deep Supervision: Four prediction vectors from three ALM groups and one global branch receive direct ground-truth supervision, then are aggregated by maximum-response voting at inference.The training objective sums weighted binary cross-entropy losses across the four branches.

4. Experiments

Across PETA, RAP, and PA-100K, the proposed framework improves pedestrian-attribute recognition through multi-scale, attribute-specific localization, with qualitative evidence showing informative regions despite occlusion and pose variation.

  • 4.2. Effectiveness of Critical Components: Adding ALMs at multiple feature levels improves mA and F1 by 3.1% and 1.3%, while removing all ALMs causes a significant performance drop.The ablation limits the framework to three feature levels and confirms the contribution of attribute-specific localization.
  • 4.2. Effectiveness of Critical Components: Feature-pyramid concatenation improves mA by 1.0%, and channel attention further raises mA to 80.61%; element-wise maximum is preferred over averaging at inference.The experiments attribute the concatenation gain to high-level top-down guidance and use deep supervision for multi-level ALMs.
  • 4.2. Effectiveness of Critical Components: The final RAP model improves over the baseline by 6.1% in mA and 1.9% in F1, with notable gains for BaldHead, Hat, and Muffler.The attribute-wise improvements are 23.1% for BaldHead, 12.4% for Hat, and 13.5% for Muffler.
  • 4.3. Visualization of Attribute Localization: ALMs localize concrete attributes into informative regions despite occlusion or pose variation, while abstract attributes use larger regions and BaldHead exposes a low-level failure case.The BaldHead failure is associated with its highly imbalanced data distribution, with only 0.4 percent of RAP images annotated for that attribute.
  • 4.4. Different Attribute-Specific Methods: Compared with attribute-specific attention and rigid-part methods, ALMs produce more accurate and discriminative regions, avoiding irrelevant attention and fixed body-part limitations.The proposed approach balances adaptive localization with bounding-box constraints that are relatively coarse but more interpretable and controllable.
  • 4.5. Comparison with State-of-the-art Methods: The proposed method outperforms existing works across all three datasets under label-based and instance-based metrics, with especially strong gains on PA-100K.On PA-100K, it improves mA and F1 by 3.7% and 1.4%, respectively, over the second-best results.

5. Conclusion

The framework automatically localizes attribute-specific regions at multiple feature levels and uses a feature pyramid to reinforce localization and region-based feature learning. Experiments on PETA, RAP, and PA-100K show significant improvements and weakly supervised discovery of informative regions.

  • Experiments on PETA, RAP, and PA-100K show that the proposed method significantly outperforms most existing methods.
  • The end-to-end framework automatically localizes attribute-specific regions at multiple feature levels without requiring region annotations.
  • A feature pyramid architecture reinforces attribute localization and region-based feature learning across feature levels.
  • The analysis indicates that the method can discover the most informative region for each attribute in a weakly supervised manner.

A. Implementation Details

The framework uses an ImageNet-pretrained BN-Inception backbone and is implemented in PyTorch with end-to-end training from image-level annotations.

  • The framework uses BN-Inception pretrained on ImageNet as its backbone and is trained end-to-end using only image-level annotations.
  • Training uses Adam with momentum 0.9, weight decay 0.0005, an initial learning rate of 0.0001, and batch size 32.
  • For RAP and PA-100K, the model is trained for 30 epochs.

B. Different Attribute-Specific Methods

The study compares attribute-specific visual attention and rigid-part localization against ALM, using multi-level localization visualizations and predefined or learned regional assignments. ALM is presented as adaptively localizing regions for individual attributes, while the alternatives use shared attention masks or externally estimated body parts.

  • ALM adaptively localizes regions and extracts regional features for each attribute, whereas comparison models use spatial attention or predefined body-part associations.
  • Attention Masks Model: The attention-masks model generates one spatial mask per attribute and feature level, then multiplies it channel-wise with the input features.
  • Rigid Parts Model: The rigid-parts model associates each attribute with predefined head, torso, legs, or whole-image regions localized using an external pose-estimation model.
  • Figure S4 provides additional localization case studies for different attributes and attribute-specific localization methods.
Loading 1910.04562v1…