Source-linked AI summary
CrowdHuman: A Benchmark for Detecting Human in a Crowd
Shuai Shao, Zijian Zhao, Boxun Li, Tete Xiao, Gang Yu, Xiangyu Zhang, Jian Sun
TL;DR
CrowdHuman addresses the limited representation of highly crowded, heavily occluded scenes in human-detection benchmarks. The paper introduces a dense, richly annotated dataset and finds that pretraining on it generalizes strongly across prior pedestrian, person, and head-detection benchmarks.
Problem
Crowd scenes are under-represented in existing human-detection benchmarks, although overlapping people make detection difficult.
Method
The paper constructs the CrowdHuman benchmark with dense crowd imagery and head, visible-region, and full-body annotations for each person.
Results
A general detection framework pretrained on CrowdHuman achieves state-of-the-art results across prior pedestrian, person, and head-detection benchmarks.
Takeaways & Limitations
CrowdHuman provides a benchmark and pretraining dataset for human-detection tasks involving crowded scenes.
Abstract
from arXiv · showhide
Human detection has witnessed impressive progress in recent years. However, the occlusion issue of detecting human in highly crowded environments is far from solved. To make matters worse, crowd scenarios are still under-represented in current human detection benchmarks. In this paper, we introduce a new dataset, called CrowdHuman, to better evaluate detectors in crowd scenarios. The CrowdHuman dataset is large, rich-annotated and contains high diversity. There are a total of $470K$ human instances from the train and validation subsets, and $~22.6$ persons per image, with various kinds of occlusions in the dataset. Each human instance is annotated with a head bounding-box, human visible-region bounding-box and human full-body bounding-box. Baseline performance of state-of-the-art detection frameworks on CrowdHuman is presented. The cross-dataset generalization results of CrowdHuman dataset demonstrate state-of-the-art performance on previous dataset including Caltech-USA, CityPersons, and Brainwash without bells and whistles. We hope our dataset will serve as a solid baseline and help promote future research in human detection tasks.
1. Introduction
CrowdHuman targets the under-represented challenge of detecting people in crowded scenes with substantial overlap and occlusion. It introduces a large, diverse benchmark with dense annotations and reports strong cross-dataset generalization.
- Motivation: Existing human-detection benchmarks under-represent crowd scenes, with low person density, limited overlap, and ignored crowded regions.Caltech-USA, COCOPersons, and CityPersons average 0.32, 4.01, and 6.47 persons per image, respectively.
- Dataset: CrowdHuman contains 15,000 training, 4,370 validation, and 5,000 testing images targeting challenging crowd scenarios.The dataset is exhaustively annotated and includes diverse scenes.
- Dataset: 470k persons across the training and validation subsets yield an average of 22.6 pedestrians per image.These statistics reflect the dataset’s substantially higher crowd density.
- Annotations: Each human instance has bound head, visible-region, and full-body bounding-box annotations.The paired annotations support multiple human-detection targets, including occluded full-body regions.
- Results: Cross-dataset experiments show state-of-the-art results on prior pedestrian, person, and head-detection benchmarks using a general detection framework without specific modification.Reported benchmarks include Caltech, CityPersons, COCOPerson, and Brainwash.
2. Related Work
Earlier pedestrian benchmarks helped advance detection but generally contain sparse scenes and under-represent crowds. Their protocols may also discard densely populated regions, limiting exhaustive evaluation of crowd detection.
- Human detection datasets: Earlier pedestrian datasets include INRIA, TudBrussels, Daimler, Caltech-USA, KITTI, and CityPersons.CityPersons was built from CityScapes and provides richer, more diverse pedestrian imagery than some earlier datasets.
- Dataset limitations: Caltech-USA and KITTI contain fewer than one person per image, while CityPersons contains approximately six.These densities indicate that crowd scenes remain relatively uncommon in the established benchmarks.
- Dataset limitations: Crowd scenes are under-represented because annotation protocols permit densely populated regions to be ignored or discarded.Exhaustive annotation of crowd regions is described as difficult and time consuming.
- Detection methods: Traditional human detectors use filters based on Integral Channel Features with a sliding-window strategy.The related work contrasts these approaches with the later predominance of CNN-based detectors.
- Detection methods: CNN-based approaches introduced deep features, multilevel architectures, multitask learning, robust NMS, and part-based models for pedestrian detection.Part-based models were specifically used to alleviate occlusion problems.
3. CrowdHuman Dataset
CrowdHuman is constructed to benchmark human detection in crowded, occluded scenes through diverse imagery, exhaustive annotations, and statistics emphasizing density, diversity, and overlap. Each person receives full-body, visible-region, and head boxes, while the dataset captures substantially more challenging crowd structure than prior benchmarks.
- Collection and annotation: CrowdHuman collects images from about 150 keywords spanning more than 40 cities, activities, and viewpoints to support diverse real-world scenes.Examples include pedestrians on Fifth Avenue, road crossings, basketball, parties, surveillance viewpoints, and horizontal viewpoints.
- Collection and annotation: Each individual is exhaustively annotated with a full bounding box, visible-region box, and head box, with annotations double-checked by another annotator.Full boxes complete invisible portions of partly occluded people; human-like objects such as statues receive an ignore label during evaluation.
- Dataset statistics: CrowdHuman contains about 340k persons and 99k ignore-region annotations across 15,000 training images, exceeding CityPersons’ volume by more than 10x.The comparison uses training-subset statistics.
- Dataset statistics: 22.6 persons per image makes CrowdHuman denser than previous datasets, while pre-filtering and annotation protocols support this higher crowdness.Caltech and KITTI average about one person per image, and CityPersons about seven.
- Dataset statistics: CrowdHuman offers broader pose and domain diversity, including about 33k identical persons, whereas Caltech, KITTI, and CityPersons are recorded from cars traversing streets.COCOPersons and CrowdHuman contain people in unlimited poses across a wide range of domains.
- Occlusion statistics: CrowdHuman represents heavier and more complex occlusion: 9 person pairs per image have IoU at least 0.3, 2.4 pairs exceed 0.5, and triple-person cases are well represented.Its occlusion distribution is more balanced than CityPersons, with 29.89% bare and 32.13% partial instances.
4. Experiments
Experiments evaluate FPN and RetinaNet on full-body, visible-body, and head detection, then test CrowdHuman pretraining across established benchmarks. CrowdHuman is challenging for occluded detection while improving cross-dataset performance.
- Experimental setup: FPN and RetinaNet serve as two baseline detectors for full-body, visible-body, and head detection on CrowdHuman.Both are based on Feature Pyramid Networks with ResNet-50 backbones.
- Evaluation metric: mMR measures average log miss rate over false positives per image, with AP and recall reported for reference.The evaluation range is 10^-2 to 10^0 false positives per image, and ignored regions are excluded.
- Detection results on CrowdHuman: FPN outperforms RetinaNet for visible-body detection, which remains difficult because visible regions vary with pose and occlusion.The paper characterizes CrowdHuman as especially challenging for state-of-the-art human detection algorithms.
- Detection results on CrowdHuman: 10.08 mMR on Caltech and 14.81 mMR on CityPersons show that CrowdHuman is more challenging than standard pedestrian benchmarks.These are FPN pedestrian-detection results evaluated on the standard reasonable set.
- Cross-dataset Evaluation: CrowdHuman pretraining improves performance on COCOPersons, Caltech, CityPersons, and Brainwash, demonstrating cross-dataset generalization.On Brainwash, pretraining further improves mMR by 2.5%.
5. Conclusion
CrowdHuman is introduced as a larger, denser human-detection benchmark with rich annotations for crowded scenes. The dataset also supports pretraining and reports strong transfer results on established detection benchmarks.
- The dataset is larger-scale and substantially more crowded than existing human-detection benchmarks.
- Each human instance has full-body, visible-region, and head bounding-box annotations.
- CrowdHuman’s rich annotations support potential visual algorithms and applications.
- The dataset serves as a pretraining resource and reports state-of-the-art results on pedestrian and head-detection benchmarks.