PaperlayerView exact revision on arXiv

Source-linked AI summary

WiderPerson: A Diverse Dataset for Dense Pedestrian Detection in the Wild

Shifeng Zhang, Yiliang Xie, Jun Wan, Hansheng Xia, Stan Z. Li, Guodong Guo

arXiv:1909.12118v1cs.CV

TL;DR

Pedestrian benchmarks lack the diversity and density needed for real-world assessment, especially in non-traffic scenes and crowded occlusions. The paper introduces WiderPerson with five annotation types, builds baseline detectors, and evaluates cross-dataset generalization. The dataset contains 13,382 images and 399,786 annotations, and detectors trained with it achieve state-of-the-art performance on Caltech-USA and CityPersons.

  • Problem

    Existing pedestrian detection benchmarks underrepresent diverse scenarios and highly occluded crowds, creating a gap between benchmark coverage and real-world requirements.

  • Method

    The paper constructs the WiderPerson dataset with diverse web-crawled images and five fine-grained annotation labels, then benchmarks improved Faster R-CNN and RetinaNet detectors.

  • Results

    Detectors trained with WiderPerson generalize across datasets and achieve state-of-the-art performance on Caltech-USA and CityPersons.

  • Takeaways & Limitations

    WiderPerson provides a dense, diverse benchmark and effective training source for pedestrian detection in varied scenarios and occlusions.

Abstract

from arXiv · show

Pedestrian detection has achieved significant progress with the availability of existing benchmark datasets. However, there is a gap in the diversity and density between real world requirements and current pedestrian detection benchmarks: 1) most of existing datasets are taken from a vehicle driving through the regular traffic scenario, usually leading to insufficient diversity; 2) crowd scenarios with highly occluded pedestrians are still under represented, resulting in low density. To narrow this gap and facilitate future pedestrian detection research, we introduce a large and diverse dataset named WiderPerson for dense pedestrian detection in the wild. This dataset involves five types of annotations in a wide range of scenarios, no longer limited to the traffic scenario. There are a total of $13,382$ images with $399,786$ annotations, i.e., $29.87$ annotations per image, which means this dataset contains dense pedestrians with various kinds of occlusions. Hence, pedestrians in the proposed dataset are extremely challenging due to large variations in the scenario and occlusion, which is suitable to evaluate pedestrian detectors in the wild. We introduce an improved Faster R-CNN and the vanilla RetinaNet to serve as baselines for the new pedestrian detection benchmark. Several experiments are conducted on previous datasets including Caltech-USA and CityPersons to analyze the generalization capabilities of the proposed dataset and we achieve state-of-the-art performances on these previous datasets without bells and whistles. Finally, we analyze common failure cases and find the classification ability of pedestrian detector needs to be improved to reduce false alarm and miss detection rates. The proposed dataset is available at http://www.cbsr.ia.ac.cn/users/sfzhang/WiderPerson

I. INTRODUCTION

Pedestrian detection benchmarks have advanced, but remain insufficiently diverse and dense for real-world assessment. WiderPerson addresses this gap with varied annotations and scenarios, alongside baseline detectors and cross-dataset evaluation.

  • Real-world pedestrian detection remains difficult because pose, scale, and occlusion vary substantially across images.
  • Existing benchmarks underrepresent non-traffic scenes and densely occluded crowds, limiting diversity and density relative to real-world requirements.Caltech-USA and KITTI contain fewer than one person per image, while CityPersons contains approximately seven.
  • WiderPerson contains 13,382 images and 399,786 annotations, averaging 29.87 annotations per image across diverse scenarios and occlusions.
  • Its five annotation labels are pedestrians, riders, partially-visible persons, crowd, and ignore regions.
  • The dataset supports training and benchmarking through public training and validation annotations, an online benchmark, and three data subsets.
  • An improved Faster R-CNN addresses density and diversity variations using finer feature maps, ignore-region and tiny-pedestrian handling, RoI enhancement, and dynamic sampling.
  • Detectors trained with WiderPerson generalize to Caltech-USA and CityPersons, achieving state-of-the-art performance on both datasets.

II. RELATED WORK

Earlier pedestrian datasets supported field progress but commonly relied on fixed traffic scenes and underrepresented crowded occlusion. WiderPerson is introduced to address both weaknesses.

  • Early pedestrian datasets were collected from automotive, static, or surveillance settings, providing varied but often constrained sources.
  • Later datasets expanded modalities and target types, including cyclists, multispectral imagery, and vehicle-mounted traffic imagery.
  • Existing datasets generally suffer limited pedestrian and background diversity because most use vehicle-mounted cameras in regular traffic scenarios.
  • Crowd scenarios and heavily occluded pedestrians remain underrepresented, producing insufficient occlusion cases for evaluation.
  • The paper aims to solve these diversity and occlusion shortcomings through a diverse and dense pedestrian detection dataset.

B. Method

Generic object detection evolved from handcrafted sliding-window methods toward CNN-based one-stage and two-stage detectors. Pedestrian detection research further emphasizes exhaustive localization and occlusion handling.

  • Traditional generic object detection used sliding windows with handcrafted features and classifiers, while CNN methods form one-stage and two-stage categories.
  • A common pedestrian-detection paradigm exhaustively scans sub-images across image locations and scales.
  • Pedestrian detection has progressed through HOG-SVM, multi-channel features, and other specialized detector designs.
  • CNN-based detectors have become predominant, with research addressing accuracy, complexity, speed, and scale-dependent pooling.
  • Occlusion handling commonly uses part-based models that learn part detectors and fuse their outputs to localize partially occluded pedestrians.
  • The paper presents WiderPerson through collection, annotation, statistical analysis, and benchmarking procedures.

A. Data Collection

WiderPerson images are collected from diverse online scenarios and annotated with five fine-grained categories using a dedicated annotation workflow. The dataset emphasizes dense people and standardized pedestrian and rider boxes.

  • A. Data Collection: 13,382 images remained after crawling approximately 50,000 candidates, removing duplicates with pHash and union-find, and filtering sparse scenes.The images were randomly split into 8,000 training, 1,000 validation, and 4,382 testing images.
  • A. Data Collection: Five annotation categories cover pedestrians, riders, partially-visible persons, crowd, and ignore regions.The annotation tool presents examples for all five types using distinct colors.
  • A. Data Collection: Pedestrian and rider boxes use the head-top and foot-middle points, then generate aligned boxes with fixed aspect ratio 0.41.Partially-visible persons, including heavily occluded individuals and unusual poses, are also boxed during annotation.
  • A. Data Collection: The annotation workflow includes thorough whole-image searches and repeated checking, with erroneous annotations re-annotated until they pass.The supplied passages describe the checking process after annotation.

D. Dataset Statistic

WiderPerson is substantially denser and more varied than established pedestrian datasets, spanning scales, image locations, identities, labels, and crowd occlusion. Its statistics characterize the benchmark as challenging for dense pedestrian detection.

  • D. Dataset Statistic: 13,382 images contain approximately 386k person and 13k ignore-region annotations, exceeding CityPersons by more than 10× in annotation count.The dataset defines Easy, Medium, and Hard difficulty by ground-truth height thresholds of 100, 50, and 20 pixels.
  • D. Dataset Statistic: 81.5%, 73.6%, and 63.4% average recall are obtained for Easy, Medium, and Hard subsets with 10,000 EdgeBox proposals per image.Lower recall indicates higher difficulty in the proposal-based analysis.
  • D. Dataset Statistic: WiderPerson covers a much wider and relatively uniform person-scale distribution, unlike the 30–100-pixel concentration in Caltech-USA and CityPersons.The scale comparison uses probability density functions over image-height bins.
  • D. Dataset Statistic: Approximately 28.87 persons appear per WiderPerson image, compared with approximately 1 in Caltech-USA and 7 in CityPersons.The paper describes WiderPerson as having much higher crowd density than previous datasets.
  • D. Dataset Statistic: WiderPerson spans varied scenarios and nearly uniform image locations, while comparison datasets are more constrained by vehicle-recorded settings and central placement.The dataset also contains approximately 236k identical-person instances and fine-grained person labels, including 29.9% partially-visible persons.
  • D. Dataset Statistic: WiderPerson averages 9.21 person pairs with IoU at least 0.3 and 2.15 pairs above 0.5 per image, indicating substantially more pairwise overlap than prior datasets.The paper frames intra-class, or crowd, occlusion as a particularly challenging detection issue.

E. Benchmarking

The benchmark releases training and validation annotations while evaluating detectors on held-out testing annotations through an online evaluation system. Performance is measured with the miss-rate metric used by Caltech-USA and CityPersons.

  • E. Benchmarking: Training and validation annotations are freely available for academic and non-profit research, while held-out testing annotations are evaluated through an online benchmark.A leaderboard maintains submitted results by name or anonymously.
  • E. Benchmarking: Riders, partially-visible persons, crowd, and ignore regions are excluded from false-negative and false-positive accounting during evaluation.Detections matching those annotations are not counted as false positives.

IV. PROVIDED BASELINE METHOD

The paper establishes Faster R-CNN and RetinaNet as baseline detectors, then improves Faster R-CNN for WiderPerson’s small, dense, and occluded pedestrians. The modifications target feature resolution, ignored regions, sampling, and RoI representation.

  • IV. PROVIDED BASELINE METHOD: Two representative baselines are built from Faster R-CNN and RetinaNet, covering two-stage and one-stage detection approaches.The stated goal is a straightforward architecture with good WiderPerson performance.
  • IV. PROVIDED BASELINE METHOD: Faster R-CNN uses an RPN to propose candidate regions, followed by Fast R-CNN classification and boundary refinement.The paper notes that vanilla Faster R-CNN underperforms on pedestrian detection because it struggles with heavily occluded and dense pedestrians.
  • IV. PROVIDED BASELINE METHOD: Removing a down-sampling operation produces a finer feature map intended to improve detection of small pedestrians.The reduced VGG-16 uses the hole algorithm to increase feature-map resolution.
  • IV. PROVIDED BASELINE METHOD: Ignore-region handling prevents potentially object-overlapping boxes from being sampled as background, while pedestrians below 20 pixels are filtered during training.Filtered pedestrians are treated as ignore regions rather than background samples.
  • IV. PROVIDED BASELINE METHOD: An SE block re-weights RoI channels according to sample-dependent informativeness before Fast R-CNN processing.The block uses global average pooling and two fully connected layers with little additional computational cost.
  • IV. PROVIDED BASELINE METHOD: A dynamic sample strategy addresses the fixed sampler’s inadequate use of positive examples in images containing approximately 28.87 persons.The strategy adjusts negative-sample selection based on the number of positive samples.

B. Vanilla RetinaNet

The experiments include a vanilla RetinaNet baseline alongside the improved Faster R-CNN, with implementation and evaluation conducted across the WiderPerson, Caltech-USA, and CityPersons datasets.

  • RetinaNet serves as the one-stage baseline, using regular dense sampling over locations, scales, and aspect ratios with focal loss for foreground-background imbalance.The focal loss down-weights well-classified examples.
  • Experiments primarily use the improved Faster R-CNN with VGG-16 unless otherwise specified.The study evaluates model improvements, dataset properties, detection results, error patterns, and generalization on Caltech-USA and CityPersons.

A. Implementation Detail.

The implementation upsamples or resizes inputs for dataset-specific image scales and trains detectors with fixed optimization schedules. Ablations evaluate architectural, sampling, annotation-handling, backbone, and anchor-design choices.

  • Data Processing: Input images are enlarged for small pedestrians, with Caltech and CityPersons scaled by 2× and 1.3×, while WiderPerson images use an 800-pixel short edge and 1400-pixel maximum long edge.Multi-scale training and testing are not applied for fair comparisons.
  • Training Setting: Improved Faster R-CNN models use dataset-specific SGD schedules, while RetinaNet is trained for 25k iterations with batch size 16 and staged learning-rate decay.Faster R-CNN training uses one TITAN X GPU and mini-batch 2; RetinaNet starts at learning rate 0.02.
  • Ablation Study: The ablations separately remove dynamic sampling, RoI feature enhancement, ignore-region and tiny-ground-truth handling, backbone reduction, and the new anchor setting.All experiments retain the same settings except for the specified component change.
  • Ablation Study: 3.39%, 2.69% and 2.36% MR reductions are reported for Easy, Medium and Hard subsets, respectively, from the new anchor setting.These results use Faster R-CNN with VGG-16 trained on WiderPerson and tested on its validation set.
  • Ablation Study: 39.62%, 46.28% and 53.26% MR become 35.12%, 42.67% and 50.17% for Easy, Medium and Hard subsets, respectively, with the finer feature map.The finer feature map provides more anchors and detailed information.

C. Dataset Analysis

Dataset analysis evaluates detection performance, data quantity and annotation quality, then examines common error modes. WiderPerson remains challenging, while more data and fine-grained annotations support detector performance.

  • Detection Results: 31.47%, 40.45% and 48.32% MR are obtained by vanilla RetinaNet on Easy, Medium and Hard subsets, respectively.The improved Faster R-CNN obtains lower MR values under both VGG-16 and ResNet-50 backbones.
  • Detection Results: 29.61%, 38.40% and 46.46% MR are obtained by improved Faster R-CNN with VGG-16 on Easy, Medium and Hard subsets, respectively.With ResNet-50, the corresponding MR values are 28.75%, 37.82% and 46.06%.
  • Detection Results: WiderPerson is described as challenging even for state-of-the-art pedestrian detection algorithms, with 5.49% MR on Caltech-USA and 12.49% MR on CityPersons.These cross-dataset results use improved Faster R-CNN with VGG-16.
  • Quantity Analysis: Detection performance improves logarithmically with training-set size across the three subsets, motivating the provision of more training data.The observation comes from randomly sampled training subsets evaluated with the baseline methods.
  • Quality Analysis: Detection performance deteriorates without ignore-region handling, supporting fine-grained rider annotations and additional ignore-region annotations.The evaluation protocol also ignores detected neighboring classes.
  • Error Analysis: Background false positives dominate among top-scoring detections, while localization errors reflect misaligned pedestrian boxes.The analysis distinguishes background mistakes from localization errors.
  • Error Analysis: Greedy NMS misses some pedestrians when their bounding boxes have high IoU, especially among small and occluded pedestrians.The passage reports these false negatives at an IoU threshold of 0.5.

D. Generalization Capability

WiderPerson pre-training improves transfer to Caltech-USA and CityPersons, although cross-domain dataset biases make fine-tuning important. The results support its diversity and generalization value.

  • Caltech-USA: 82.79% MR results when a Caltech-USA-trained model is evaluated on WiderPerson Easy, compared with 9.72% without fine-tuning and 4.27% with fine-tuning after WiderPerson training.The comparison is attributed to Caltech-USA’s limited density and diversity.
  • Caltech-USA: WiderPerson pre-training improves Caltech-USA performance and reaches performance described as on-par with state-of-the-art methods.The reported evaluation uses the Caltech-USA reasonable setting.
  • CityPersons: 73.45% MR occurs when training on CityPersons and testing on WiderPerson Easy, whereas WiderPerson training yields 16.17% MR on CityPersons validation.The passage presents this as evidence of WiderPerson’s difficulty and expandability.
  • CityPersons: WiderPerson pre-training reduces CityPersons MR from 12.49% to 11.13%, described as on-par with state-of-the-art results.The result uses the improved Faster R-CNN model.
  • Summary: Cross-dataset pre-training improves performance on both benchmarks, while training solely on another domain without fine-tuning performs worse because of dataset biases.The summary attributes the gains to higher diversity and retained general task features.

VI. CONCLUSION

WiderPerson addresses gaps in benchmark diversity and density by providing a large dataset spanning varied scenarios and occlusions. It also supplies baseline detectors and supports cross-dataset evaluation and training.

  • WiderPerson contains 13,382 images and 399,786 annotations spanning substantial variation in scenario and pedestrian occlusion.The dataset is intended to provide high-quality annotations for dense pedestrian detection in the wild.
  • The dataset enables experiments for training better pedestrian detectors and evaluating them as a new test benchmark.
  • Baseline detectors based on Faster R-CNN and RetinaNet are proposed for benchmarking state-of-the-art pedestrian detectors.
  • Cross-dataset results show WiderPerson is an effective training source and helps achieve state-of-the-art performance on Caltech-USA and CityPersons.
Loading 1909.12118v1…