Source-linked AI summary
A Global-Local Emebdding Module for Fashion Landmark Detection
Sumin Lee, Sungchan Oh, Chanho Jung, Changick Kim
TL;DR
Fashion landmark locations vary across poses, scales, and clothing styles, making contextual understanding important for accurate detection. The paper combines non-local and convolution operations in a global-local embedding module, and reports that its network outperforms state-of-the-art methods on two benchmark datasets.
Problem
Fashion landmarks vary spatially across poses, scales, and styles, requiring comprehensive clothing context for accurate detection.
Source: §1 · p. 1Method
The network uses a global-local embedding module that combines a non-local operation for long-range dependencies with convolutions for local representation.
Source: §1 · p. 1, §2.2 · pp. 2–3Results
Takeaways & Limitations
The authors conclude that the method improves clothing feature representation for fashion landmark detection.
Source: §4 · p. 4
Abstract
from arXiv · showhide
Detecting fashion landmarks is a fundamental technique for visual clothing analysis. Due to the large variation and non-rigid deformation of clothes, localizing fashion landmarks suffers from large spatial variances across poses, scales, and styles. Therefore, understanding contextual knowledge of clothes is required for accurate landmark detection. To that end, in this paper, we propose a fashion landmark detection network with a global-local embedding module. The global-local embedding module is based on a non-local operation for capturing long-range dependencies and a subsequent convolution operation for adopting local neighborhood relations. With this processing, the network can consider both global and local contextual knowledge for a clothing image. We demonstrate that our proposed method has an excellent ability to learn advanced deep feature representations for fashion landmark detection. Experimental results on two benchmark datasets show that the proposed network outperforms the state-of-the-art methods. Our code is available at https://github.com/shumming/GLE_FLD.
1. Introduction
Fashion landmarks describe clothing structures, but their locations vary widely across poses, scales, and styles, making contextual understanding important for accurate detection. The proposed network combines global and local context and outperforms state-of-the-art methods on two benchmark datasets.
1. Introduction
- Fashion landmarks are key-points describing structures such as collars, sleeves, waistlines, and hemlines; landmark counts differ across clothing types.Source: §1 · p. 1, Figure 1 · p. 1
- Landmarks vary spatially across clothing poses, scales, and styles, so accurate detection requires comprehensive semantic information about clothes.Source: §1 · p. 1
- A non-local operation captures long-range dependencies, followed by convolutions that enhance local representation and provide both global and local clothing context.The resulting features are used to predict high-resolution landmark heatmaps.Source: §1 · p. 1
- The network uses VGG-16 for feature extraction, a global-local embedding module for landmark features, and an upsampling network to predict heatmaps.Source: Figure 2 · p. 2
- The proposed network outperforms state-of-the-art methods on the DeepFashion and FLD datasets.The introduction reports experiments on both datasets.Source: §1 · p. 2
2. Methods
The network extracts clothing features, embeds global and local context, then upsamples them into input-sized landmark heatmaps. Its embedding module combines non-local dependencies with local convolutional processing.
2.1. Network architecture
- Input-sized landmark heatmaps assign each location a probability of containing a landmark.The network uses transposed convolutions to enlarge feature maps and a 1×1 convolution to produce a heatmap for each landmark.Source: §2.1 · p. 2
2.2. Global-local embedding module
- After VGG-16 conv4 3, the global-local embedding module generates rich landmark embedding features to exploit contextual knowledge of clothing items and consists of a non-local block and two convolutional layers.Source: §2.2 · p. 2, §2.1 · p. 2
- The non-local operation computes long-range dependencies between image points and sums weighted input features.Its embedded Gaussian function uses a normalization factor, and θ, φ, and g are 1×1 convolutions.Source: §2.2 · p. 2
- A residual connection adds the input to the projected non-local output, yielding features with global clothing context.The projection w is a 1×1 convolution.Source: §2.2 · p. 2
- Two convolutional layers perform weighted sums over local neighborhoods, giving the output feature both global and local context.Each layer includes a 3×3 convolution, batch normalization, and ReLU.Source: §2.2 · p. 3
- The network stacks two embedding modules for Deepfashion and three for FLD.Source: §2.2 · p. 3
3. Experiments
Experiments evaluate the network on DeepFashion and FLD using normalized error, and report better performance than competing methods on both datasets. Qualitative examples show right- and left-side landmark discrimination in back- and side-view images.
3.1. The benchmark datasets and evaluation
- DeepFashion and FLD provide training, validation, and test splits, and evaluation uses normalized error: landmark L2 distance in normalized coordinates.DeepFashion contains 289,222 images and FLD contains 123,016; both datasets include bounding boxes and landmarks.Source: §3.1 · p. 3
3.2. Results
- The network outperforms all compared methods, with normalized error of 0.0568 on FLD and 0.0393 on DeepFashion.The authors also report improvements for waistline and hem landmarks, which are difficult for other methods to discriminate.Source: §3.2 · p. 3
- Qualitative results show the model distinguishing right- and left-side landmarks in back- and side-view images.The authors describe the network as robust to viewpoint variations and deformations.Source: §3.2 · p. 3
4. Conclusion
The paper concludes that its global-local embedding module combines non-local and convolution operations to use global and local clothing context. The proposed method achieves state-of-the-art performance on two benchmark datasets.
4. Conclusion
- The proposed work is a fashion landmark detection network with a global-local embedding module.Source: §4 · p. 3
- The global-local embedding module combines non-local and convolution operations to exploit both global and local clothing context.The paper states that this module improves clothing feature representations for fashion landmark detection.Source: §4 · p. 4
- The method achieves state-of-the-art performance on two benchmark datasets.Figure 3 presents qualitative examples from DeepFashion and FLD, placing predictions beside ground-truth landmarks.Source: §4 · p. 4, Figure 3 · p. 4