Source-linked AI summary
A Richly Annotated Dataset for Pedestrian Attribute Recognition
Dangwei Li, Zhang Zhang, Xiaotang Chen, Haibin Ling, Kaiqi Huang
TL;DR
Pedestrian attribute recognition needs better benchmarks for real surveillance because existing datasets are limited or heterogeneous and rarely annotate viewpoint, occlusion, and body-part conditions. The paper constructs RAP, a large multi-camera dataset with fine-grained and contextual annotations, and evaluates recognition with baseline models and multi-label metrics. The authors report that contextual information can improve attribute recognition in real applications.
Problem
Existing pedestrian attribute datasets are limited or heterogeneous and generally lack annotations for environmental and contextual factors affecting recognition.
Method
The paper builds RAP from real surveillance scenarios and evaluates SVM and CNN baselines using multi-label performance metrics and contextual-factor analyses.
Results
41,585 pedestrian images with 72 attributes are collected, and quantitative analysis shows that viewpoint, occlusion, and body-part guidance can produce impressive improvements.
Takeaways & Limitations
RAP provides a benchmark foundation for large-scale attribute recognition in real surveillance scenarios and supports analysis of contextual factors.
Abstract
from arXiv · showhide
In this paper, we aim to improve the dataset foundation for pedestrian attribute recognition in real surveillance scenarios. Recognition of human attributes, such as gender, and clothes types, has great prospects in real applications. However, the development of suitable benchmark datasets for attribute recognition remains lagged behind. Existing human attribute datasets are collected from various sources or an integration of pedestrian re-identification datasets. Such heterogeneous collection poses a big challenge on developing high quality fine-grained attribute recognition algorithms. Furthermore, human attribute recognition are generally severely affected by environmental or contextual factors, such as viewpoints, occlusions and body parts, while existing attribute datasets barely care about them. To tackle these problems, we build a Richly Annotated Pedestrian (RAP) dataset from real multi-camera surveillance scenarios with long term collection, where data samples are annotated with not only fine-grained human attributes but also environmental and contextual factors. RAP has in total 41,585 pedestrian samples, each of which is annotated with 72 attributes as well as viewpoints, occlusions, body parts information. To our knowledge, the RAP dataset is the largest pedestrian attribute dataset, which is expected to greatly promote the study of large-scale attribute recognition systems. Furthermore, we empirically analyze the effects of different environmental and contextual factors on pedestrian attribute recognition. Experimental results demonstrate that viewpoints, occlusions and body parts information could assist attribute recognition a lot in real applications.
1 Introduction
Pedestrian attribute recognition matters for surveillance applications but remains difficult because existing datasets are limited, heterogeneous, and weakly annotated for environmental factors. RAP addresses these gaps with a large, richly annotated surveillance dataset, empirical factor analysis, baseline systems, and multi-label evaluation metrics.
- Human attributes support surveillance tasks including detection, person re-identification, face recognition, and people search.
- Existing datasets have limited samples or sparse annotations, while PETA combines 19,000 images with 65 attributes from 10 small datasets.
- 41,585 RAP samples from 26 camera scenes and long-term surveillance collection are annotated with 72 fine-grained attributes.
- RAP additionally annotates viewpoints, occlusion styles, and body parts to support analysis of environmental and contextual influences.
- Baseline SVM and CNN systems are evaluated, while accuracy, precision rate, recall rate, and F1 value are introduced for multi-label benchmark evaluation.
2 Related Work
Prior pedestrian attribute recognition largely uses full-body features and classifiers, with CNNs enabling joint prediction of multiple attributes. Part-based work addresses viewpoint and occlusion, but large-scale analyses of these factors remain absent because existing datasets are too small, heterogeneous, or insufficiently annotated.
- Earlier methods used low-level features, SVMs, latent SVMs, and intersection-kernel SVMs for pedestrian attribute recognition.
- CNN approaches such as ACN and DeepMAR jointly model multiple attributes using holistic full-body representations.
- PANDA uses pose-aligned part networks and combines their features to address viewpoint and occlusion problems.
- No previous large-scale surveillance study had analyzed viewpoint, occlusion, and body-part influences on attribute recognition.
- This gap reflects both the small scale of surveillance attribute datasets and the heterogeneity and missing factor annotations of larger datasets such as PETA and APiS.
3 The RAP dataset
RAP is collected from a multi-camera shopping-mall surveillance network and combines fine-grained attribute labels with spatial, contextual, occlusion, viewpoint, and coarse body-part annotations. Its design reflects practical surveillance constraints, including small, blurred pedestrians and difficult fine-grained part localization.
- RAP uses 26 cameras from a 1,280×720 shopping-mall surveillance network.
- Three consistent months of video at 15 frames per second were collected, then detection and tracking selected samples for annotation.
- The annotations cover spatial-temporal information, whole-body attributes, accessories, postures and actions, occlusion, and body-part attributes.
- RAP adds viewpoints, occlusions, human parts, and fine attributes beyond typical pedestrian attribute datasets.
- Viewpoints: Four viewpoint categories are labeled by full-body direction: front, back, left, and right.
- Occlusions and body parts: Occlusions are labeled by position and source, while body parts use three coarse regions—head-shoulder, upper body, and lower body—because surveillance pedestrians are small and blurred.
4 Evaluation methods and benchmark systems
The paper evaluates pedestrian attribute recognition with both label-based and example-based criteria, then benchmarks SVM and CNN-based systems using handcrafted and learned features.
- Evaluation methods: Mean accuracy averages positive and negative classification accuracy for each attribute, then averages across attributes.This criterion addresses unbalanced attribute distributions by treating positive and negative examples separately.
- Evaluation methods: Example-based evaluation measures prediction consistency per pedestrian image using accuracy, precision, recall, and F1.It complements label-based evaluation by accounting for inter-attribute correlations within each example.
- Benchmark systems: SVM baselines use a linear kernel with ELF, FC6, or FC7 features; regularization C is selected by cross-validation.The best reported C values differ by feature type: 1 for ELF and 0.1 for CNN features.
- Benchmark systems: Two multi-attribute CNN baselines, ACN and DeepMAR, are implemented with CaffeNet-based architectures and joint learning.Both models add attribute-specific fully connected layers or use different loss functions, with training settings specified for each model.
5 Experiments
Experiments evaluate baseline attribute recognition and examine how viewpoints, occlusions, body parts, attribute imbalance, and evaluation choices affect performance. The analyses show that contextual annotations and example-based evaluation reveal important differences in recognition behavior and prediction consistency.
- Experimental setup: Five random splits use 33,268 training images, 8,317 test images, and 51 selected binary attributes for evaluation.Attributes are selected when positive examples exceed 0.01, with baldhead additionally included.
- Overall evaluation: CNN FC6 features outperform ELF, while FC6 generalizes better than FC7 for baseline attribute recognition.Gender, backpack, role, trousers, and pushing-action attributes achieve comparatively higher recognition accuracy.
- Viewpoint: Viewpoint affects attribute recognition, with Cleark, Holding, and Hat especially easy to recognize from the front viewpoint.Viewpoint classification with FC6 performs best on mixture data across all four viewpoints; removing occluded data improves accuracy by 1.2%.
- Occlusion: A 5.6% drop in average accuracy shows that part-corresponding attributes are more affected by occlusion than full-body attributes.BodyFat changes little, whereas attributes such as SportsShoes, located on the lower body, are more vulnerable.
- Body parts: Part-based recognition aligns attributes with relevant regions: Female is strongest from Fullbody, LongHair from HeadShoulder, and Suit-up and Backpack from UpperBody features.The experiments report UpperBody recognition as easier than using two parts or full-body information alone.
- Evaluation and dataset comparison: Example-based evaluation substantially improves ACN and DeepMAR results, with joint learning exceeding SVM by more than 25% in Accuracy and Precision.On RAP, label-based performance can stagnate or decline because many attributes are extremely imbalanced; PETA results are generally higher and may include the same person across splits.
6 Conclusions
The paper introduces RAP to strengthen pedestrian attribute recognition in real surveillance scenarios. Its richly annotated data and benchmark analyses support studying how contextual factors affect recognition.
- RAP contains 41,585 pedestrian images with annotations for viewpoints, occlusions, body parts, and common attributes.The dataset is designed for developing large-scale attribute recognition algorithms.
- The benchmark evaluates linear SVM, ACN, and DeepMAR models for comparison on the RAP dataset.
- Quantitative analyses examine viewpoints, occlusions, body parts, and attribute interrelationships in recognition.
- Environmental- and contextual-factor-guided recognition obtains impressive improvements in real applications.