PaperlayerView exact revision on arXiv

Source-linked AI summary

Attribute-Aware Attention Model for Fine-grained Representation Learning

Kai Han, Jianyuan Guo, Chao Zhang, Mingjian Zhu

arXiv:1901.00392v2cs.CV

TL;DR

Fine-grained recognition must distinguish small visual differences, while prior methods often focus on metrics or ensembles and lack local information. A3M uses reciprocal attention to learn local attribute and global category representations together. The authors report effectiveness across four benchmarks and several recognition tasks.

  • Problem

    Fine-grained recognition seeks discriminative representations despite small visual differences, and many prior methods emphasize metrics or ensembles while lacking local information.

    Source: §1 · pp. 1–2

  • Method

    A3M learns local attribute and global category representations together using reciprocal attention to select relevant regional category and attribute features.

    Source: §1 · p. 2, §3.2 · p. 4, §3.3 · p. 4

  • Results

    The authors report A3M's effectiveness across person re-identification, fine-grained classification, and retrieval on Market-1501, CompCars, CUB-200-2011, and CARS196.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    For retrieval with category information but no attribute labels, A3M combines local and global features without time-consuming ensembles or complicated losses.

    Source: §1 · p. 2

Abstract

from arXiv · show

How to learn a discriminative fine-grained representation is a key point in many computer vision applications, such as person re-identification, fine-grained classification, fine-grained image retrieval, etc. Most of the previous methods focus on learning metrics or ensemble to derive better global representation, which are usually lack of local information. Based on the considerations above, we propose a novel Attribute-Aware Attention Model ($A^3M$), which can learn local attribute representation and global category representation simultaneously in an end-to-end manner. The proposed model contains two attention models: attribute-guided attention module uses attribute information to help select category features in different regions, at the same time, category-guided attention module selects local features of different attributes with the help of category cues. Through this attribute-category reciprocal process, local and global features benefit from each other. Finally, the resulting feature contains more intrinsic information for image recognition instead of the noisy and irrelevant features. Extensive experiments conducted on Market-1501, CompCars, CUB-200-2011 and CARS196 demonstrate the effectiveness of our $A^3M$. Code is available at https://github.com/iamhankai/attribute-aware-attention.

1 INTRODUCTION

Fine-grained recognition is difficult because small visual differences can be obscured by changes in pose, lighting, or viewpoint, while existing part-based approaches have annotation or localization drawbacks. A3M combines attribute and category features through reciprocal attention, and the authors report results across re-identification, classification, and retrieval benchmarks.

  • Fine-grained recognition is challenging because small differences between categories or instances can be overwhelmed by changes in location, pose, lighting, or viewpoint.Manually annotated parts are laborious and may be suboptimal, while weak-label methods may not locate accurate parts.Source: §1 · p. 1
  • Attributes such as a handbag can distinguish otherwise similar people, while gender or upper-clothing color can help distinguish other individuals.The examples motivate using local visual details for fine-grained recognition.Source: §1 · pp. 1–2
  • A3M learns local attribute features and global category features simultaneously, using reciprocal attention to select relevant category and attribute features.Attribute cues refine category features, and the relative importance of attributes can vary across categories.Source: Figure 1 · p. 1, §1 · p. 2
  • A3M is also used for fine-grained retrieval with category information but no attribute labels, combining local and global features without time-consuming ensembles or complicated losses.Source: §1 · p. 2
  • The authors report superiority across person re-identification, fine-grained classification, and retrieval on Market-1501, CompCars, CUB-200-2011, and CARS196.Source: §1 · p. 2

2 RELATED WORK

Prior work explores hand-crafted features, learned representations, parts, attributes, metric learning, and attention for fine-grained recognition and re-identification. The paper positions A3M as using attention to select and fuse global category and local attribute features.

  • Person re-identification methods include hand-crafted color and texture features with metric learning, as well as end-to-end deep representations and distance metrics.Prior work is described as limited by insufficient use of attribute and regional features.Source: §2 · p. 2
  • Fine-grained recognition methods use bilinear features, predefined or weakly discovered parts, object-part attention, dictionaries, and category-specific part models.The passage notes that predefined-part methods rely heavily on manual definitions and that an earlier attribute-guided method hard-crops parts.Source: §2 · p. 2
  • Prior attribute-based approaches use attribute losses or triplet losses, while the paper proposes attention to select better attribute features.The related-work passage says these approaches do not further explore feature importance or relationships between attributes and categories.Source: §2 · p. 2
  • Triplet-based retrieval methods impose similarity constraints but may require hard-example mining over Θ(N^3) training data; ensemble methods use multiple models at additional time and memory cost.Source: §2 · pp. 2–3
  • The paper uses soft attention to generate attentions for fusing global category features and local attribute features simultaneously.Soft attention computes weighted feature combinations to focus on task-relevant parts.Source: §2 · p. 3

3 ATTRIBUTE-AWARE ATTENTION MODEL

Section 3 describes an end-to-end model with separate attribute- and category-feature branches, whose reciprocal attention selects relevant regions and attributes. The selected category and attribute features are combined for final category recognition.

3.1 Initial Category and Attribute Recognition

  • The network first introduces separate branches to extract attribute features and category features, which help one another select features.Source: §3 · p. 3

3.1.1 Shared CNN.

  • The shared CNN extracts high-level features from local image regions; its feature map is shared by the subsequent branches.The shared CNN is ImageNet-pretrained; the example ResNet-50 output is 2048 × 7 × 7 for a 224 × 224 × 3 input.Source: §3.1.1 · p. 3

3.1.2 Category Branch.

  • The category branch transforms the shared map into local category vectors and pools them into a global category embedding for softmax classification.Category classification is trained with cross-entropy loss.Source: §3.1.2 · p. 3

3.1.3 Attribute Branch.

  • For each attribute, the attribute branch produces an embedding and uses softmax classification; its training loss encourages attention to related regions.Source: §3.1.3 · pp. 3–4

3.2 Attribute-Category Reciprocal Attention

  • Attribute-guided attention selects category-map regions related to intrinsic attributes, while category-guided attention weights attributes relevant to category recognition.The attribute-guided module merges its K regional attention maps by max pooling, then weights and sums regional features to produce a category representation.Source: §3.2 · p. 4

3.3 Final Category Recognition

  • The final representation adds the weighted category and attribute features, combining category and intrinsic-attribute information after reciprocal selection.Final category classification uses cross-entropy loss.Source: §3.3 · p. 4

4 EXPERIMENTS

The experiments evaluate A3M on person re-identification, fine-grained classification, and image retrieval, alongside dataset and implementation details. Results report improvements over baselines or prior methods across these tasks, with ablations and visualizations examining the attention modules.

4.3 Evaluations on Person Re-identification

4.1 Datasets

  • The experiments use Market-1501, CUB-200-2011, CompCars, and CARS196, with differing splits and attribute or part annotations across tasks.For retrieval on CUB-200-2011, attributes are not used; the first 100 classes train and the remainder test.Source: §4.1.1 · p. 5, §4.1.2 · p. 5, §4.1.3 · p. 5

4.2 Implementation Details

  • The shared CNN is ImageNet-pretrained and fine-tuned with SGD for 60 epochs; inputs are resized by dataset, and classification and re-identification use 512-dimensional embeddings.Retrieval uses 128- and 384-dimensional final fully connected layers; α and β are set to 0.5.Source: §4.2 · pp. 5–6

4.3 Evaluations on Person Re-identification

  • On Market-1501, either attention module outperforms Baseline-2, and combining both gives the best results; maps highlight attribute-related regions, though sleeve length is harder to learn.Visualized maps associate gender, bag, upper-clothing color, sleeve length, and lower-clothing color with different regions.Source: Figure 5 · p. 5, §4.3 · p. 6

4.4 Evaluations on Image Classification

  • A3M achieves competitive best accuracy on CUB-200-2011 and CompCars, with at least 1.3% accuracy gain over baselines.The tables identify results with attribute or parts annotations used during training.Source: Table 2 · p. 6, Table 3 · p. 6, §4.4 · pp. 6–7
  • CUB-200-2011 performance improves as α or β rises from 0.1 to 0.5, but further increasing α decreases performance; the authors report low sensitivity to these parameters.A3M also has higher overall attribute-classification accuracy than Baseline-2 on CUB-200-2011.Source: §4.4 · p. 7, Figure 7 · p. 7, Figure 8 · p. 8

4.5 Evaluations on Image Retrieval

  • 61.2% Recall@1 on CUB-200-2011 exceeds reproduced HDC's 60.0%, while CARS196 results improve around 1% over baseline-1 on Recall@{1,2}.The retrieval tables report Recall@K and NMI for CUB-200-2011 and CARS196.Source: §4.5 · pp. 7–8, Table 4 · p. 8, Table 5 · p. 8

5 CONCLUSION

A3M learns global category and local attribute features for person re-identification, fine-grained classification, and retrieval. Its reciprocal attention selects important features, and experiments show effectiveness on fine-grained recognition problems.

  • A3M learns global category features and local attribute features for person re-identification, fine-grained classification, and retrieval.Source: §5 · p. 8
  • Reciprocal attention uses category and attribute features to select important regional category and attribute features.Source: §5 · p. 8
  • Experiments show A3M is effective on fine-grained image recognition problems.Source: §5 · p. 8

Related papers

Loading 1901.00392v2…