Locate, Size and Count: Accurately Resolving People in Dense Crowds via Detection

1 INTRODUCTION

• Crowd-counting models must handle substantial diversity in the appearance of individual people and their assemblies, including variation in pose, viewpoint, and illumination within the same crowd as well as across crowd images.Source: §1 · p. 1
• LSC-CNN treats crowd counting as dense detection, locating each person’s head with a bounding box and deriving the count from detections.Unlike density regression, detections can pinpoint people for applications such as recognition and tracking.Source: §1 · pp. 1–2
• Its fine-grained, high-resolution CNN uses top-down feature modulation to jointly process multiple scales and better resolve people.The framework is tailored to crowd scenes rather than relying on conventional detector changes.Source: §1 · pp. 1–2
• A practical training regime uses only point head annotations yet dynamically estimates head-box sizes for detection.Crowd datasets lack the bounding-box annotations needed to train ordinary detectors.Source: §1 · p. 2
• A winner-take-all loss is proposed for high-resolution training, alongside pixel-wise classification with cross-entropy rather than density-estimation L2 regression.The introduction identifies local minima as a risk when high-resolution gradients are averaged across large spatial areas.Source: §1 · p. 2
• The introduction characterizes the benchmarked LSC-CNN as delivering impressive localization, sizing, and counting performance.No quantitative comparison is supplied in these passages.Source: §1 · p. 2

Person Detection

• Person-detection methods use appearance features or motion vectors (,,), but generally struggle in highly dense crowds and require bounding-box training annotations.Idrees et al. use local scale priors and global occlusion reasoning, while [19] use deep features in a recurrent detection-and-counting framework.Source: §2 · p. 2

Density Regression

• Density-regression methods use head detections, interest points, and frequency analysis or CNN regression, while direct count regression lacks spatial information in its loss.Source: §2 · p. 2

Multiple and Multi-column CNNs

• Multi-network and multi-column approaches address crowd appearance diversity through cascaded correction, scale-specific fusion, specialized receptive fields, and other expert architectures,,,.The approaches include three density-specialized MCNN columns, differential column training, hierarchical experts, and parallel convolutional blocks with consistency loss.Source: §2 · p. 3

Leveraging context and other information

• Context-enhanced density regression supplies auxiliary local or global information (,) or uses top-down feedback and incremental refinement to improve predictions.Source: §2 · p. 3

Better and easier architectures

• Decide-Net uses Faster R-CNN predictions in sparse regions, but its dense-crowd performance is unclear and it requires some box annotations; other work pursues simpler regression architectures,.The cited simpler models use VGG with dilated convolutions or DenseNet with multi-resolution density regression and composition loss.Source: §2 · p. 3

Other counting works; WTA Architectures

• Alternative counting methods use unlabeled data for count-ranking self-supervision or Grid Winner-Take-All features, while other approaches use Negative Correlation Learning and adversarial training.LSC-CNN instead predicts head bounding boxes; its GWTA training restricts gradient updates to the highest-loss spatial region, following.Source: §2 · pp. 3–4

Object/face Detectors

• Unlike general object and face detectors (–,,,), LSC-CNN predicts at higher resolutions for tiny, adjacent heads and classifies predefined boxes rather than regressing their sizes.The paper notes that dense-crowd box annotations are unavailable and must rely on pseudo data; it also describes dense crowd analysis as harder due to large diversity.Source: §2 · p. 3

A concurrent work

• Unlike concurrent PSDNN, LSC-CNN dynamically selects boxes from head locations, avoids box regression and ground-truth updating, and reportedly outperforms it on almost all benchmarks.PSDNN uses Faster R-CNN with pseudo ground truth from point annotations, a locally constrained regression loss, and iterative box updates; LSC-CNN is end-to-end and single-stage.Source: §2 · p. 3

• LSC-CNN formulates locating and sizing people as an end-to-end, single-stage process that predicts localized head boxes.Source: §3 · p. 4
• A Feature Extractor produces multiresolution feature maps, which Top-down Feature Modulator networks fuse to make box predictions.Source: §3 · p. 4
• Non-Maximum Suppression selects valid detections from multiple resolutions for the final output; training instead uses a GWTA Loss module with WTA backpropagation and adaptive ground-truth box selection.Source: §3 · p. 4

3.1.1 Feature Extractor

• VGG-16’s first five convolution blocks form a 224×224 RGB-input backbone, branching into feature maps at one-half, one-fourth, one-eighth, and one-sixteenth resolution.Copied blocks share initialization but not weights, allowing branches to specialize; Figure 3 depicts the modified feature extractor.Source: §3.1.1 · p. 4, Figure 3 · p. 4
• The branches address crowd diversity and scale variation, using high-resolution features for tightly packed people and lower-resolution, larger-receptive-field features for sparser crowds.The paper says increased branch resolution helps resolve nearby people, while one-sixteenth features can discriminate people appearing large in very sparse crowds.Source: §3.1.1 · p. 4

3.1.2 Top-down Feature Modulator

• Because high-resolution features can mistake people-like clutter for heads, each scale’s Top-down Feature Modulator combines its features with outputs from lower-resolution branches to add global context and suppress spurious detections.Each TFM acts as a detector for its scale; the lowest-resolution TFM receives no top-down features, while higher-resolution modules receive outputs from all preceding lower-resolution TFMs.Source: §3.1.2 · p. 5
• TFM fuses upsampled top-down features with scale features and classifies each pixel as background or one of the predefined head boxes, typically using three box sizes per branch.The fusion layers reduce channels before concatenating the processed inputs; the resulting predictions support fine-grained spatial and scale localization, with branches focusing on appropriate box-size subsets. Figure 4 illustrates the module.Source: §3.1.2 · p. 5, Figure 4 · p. 5

3.2.1 Box classification

• Per-pixel box classification replaces box-size regression, handling approximate labels and extreme head-size variation without requiring size annotations beyond head points.The model classifies heads into predefined box classes or background; the authors argue this avoids the normalization and data-balancing complexity associated with regressing extreme size ranges.Source: §3.2.1 · pp. 5–6
• Head box sizes are approximated by each annotation’s nearest-neighbor distance, discretized into predefined bins, with non-person locations assigned to background.The authors note this approximation can be inaccurate in sparse crowds, but report that it works fairly well across a wide range of densities; a lone person receives an infinite pseudo-size.Source: §3.2.1 · p. 6
• Box sizes use finer increments at higher resolutions and coarser increments at lower resolutions; the typical increments are γ = {4, 2, 1, 1}.The highest-resolution branch fixes its first box size to one to improve resolution for highly dense crowds.Source: §3.2.1 · p. 6

3.2.2 GWTA Training

• Training weights are derived from train-set class frequencies to balance branches and classes, with box-class weights scaled relative to background and capped at 10.The frequency-based weighting addresses severe imbalance, including the skew toward background pixels.Source: §3.2.2 · pp. 6–7
• GWTA addresses local-minimum risk by computing loss in the highest-loss grid cell at each scale, emphasizing difficult regions instead of averaging gradients across the full map.The cell dimensions are based on the lowest-resolution prediction map and grow by a factor of four at subsequent scales.Source: §3.2.2 · p. 7

3.3.1 Prediction Fusion

• All branches produce predictions at multiple resolutions, whose box locations are extracted and linearly scaled to the input resolution.At testing, the GWTA training module is replaced with prediction fusion.Source: §3.3.1 · p. 7
• Standard NMS removes boxes with overlap above a threshold; the remaining boxes form the final prediction and are enumerated to produce the crowd count.Source: §3.3.1 · p. 7
• The NMS threshold is 0.3 for intermediate training evaluations, while the best trained model uses a validation-set search to minimize MAE, typically with thresholds from 0.2 to 0.3.Source: §3.3.1 · p. 7

4.1 Experimental Setup and Datasets

• Evaluation covers major crowd datasets for localization and counting, WIDERFACE for box sizing, and TRANCOS for vehicle-counting generalization.Because crowd datasets provide point annotations rather than boxes, sizing is benchmarked on WIDERFACE. For UCF CC 50, the model uses only the one-eighth and one-fourth branches with five-fold cross-validation; WorldExpo’10 uses the one-sixteenth and one-eighth branches, while TRANCOS uses WorldExpo’10’s architecture and box sizes.Source: §4.1 · pp. 7–8

4.2 Evaluation of Localization

• LSC-CNN localizes people more accurately than CSR-A on GAME and MLE, although thresholding CSR-A to obtain detections raises its MAE from 72.6 to 167.1.GAME(0) equals MAE; MLE matches predictions one-to-one with head annotations and applies a 16-pixel penalty to absent or spurious detections.Source: §4.2 · p. 9, Table 1 · p. 9

4.3 Evaluation of Sizing

• Trained on pseudo boxes from WIDERFACE point annotations, LSC-CNN beats the CSR-A-thr baseline and has higher mAP than PSDNN on the Medium and Hard sets.Its mAP is weaker on Easy, where large faces expose the limits of discretized box sizes; training with actual box annotations improves performance. Box classification accuracy is around 94.56% on ST PartA and 93.97% on UCF-QNRF.Source: §4.3 · p. 9, Table 2 · p. 9

4.4 Evaluation of Counting

• 120.5 MAE on UCF-QNRF, 12.5 below, and LSC-CNN achieves state-of-the-art on Shanghaitech Part B and UCF CC 50.On Shanghaitech Part A, MAE is just 0.5 above PSDNN, while UCF CC 50 MAE falls by 33 points; LSC-CNN also beats average MAE on WorldExpo’10 and is below PSDNN on TRANCOS, near the best method.Source: §4.4 · pp. 9–10, Table 3 · p. 9, Table 4 · p. 9

5.1 Effect of Multi-Scale Box Classification

• Counting accuracy improved as scale branches increased from nS = 1 to nS = 4, slightly worsened at nS = 5, and gains from box classes saturated after nB = 3.The authors attribute the nS = 5 error increase to more trainable parameters for the same data, and the nB saturation to fewer training samples per box class.Source: §5.1 · p. 10, Table 7 · p. 10

5.2 Architectural Ablations

• Multiple top-down TFM connections improved count accuracy more than sequential connections, while sigmoid gating caused only a slight performance drop versus LSC-CNN.Removing top-down connections also reduced performance; the authors suggest top-down modulation can help in different forms. The proposed concatenation of top-down and bottom-up features performed best among the described alternatives.Source: §5.2 · p. 10, Table 8 · p. 10

5.3 Comparison with Object/Face Detectors

• Existing detectors had poor counting performance from high MAE in dense crowds, while LSC-CNN produced convincing detections across density ranges.FRCNN, SSH, and TinyFaces were trained on crowd datasets using adjusted anchors and pseudo-box ground truth; evaluation included MAE, MSE, and MLE. Inference time and model size were also compared, with inference time averaged over the ShanghaiTech Part A test set on an NVIDIA V100 GPU.Source: Table 9 · p. 11, Figure 9 · p. 11, §5.3 · p. 11

5.2 Architectural Ablations

• Without GWTA, LSC-CNN failed on the one-half-resolution branch and showed a significant MAE drop, supporting GWTA’s role in model optimization.Figure 8 highlights the high-resolution branch failure, which the authors associate with gradient averaging.Source: §5.2 · pp. 10–11, Figure 8 · p. 10
• Removing class weighting produced unacceptable counting error, while replacing per-pixel box classification with box-size regression failed to achieve good MAE.The authors attribute the unweighted model’s error mainly to skewed person counts across scales and suggest class imbalance across box sizes may explain the regression result.Source: Table 8 · p. 10, §5.2 · p. 11

6 CONCLUSION

• LSC-CNN achieves better crowd counting than existing regression methods and superior localization while retaining the merits of a detection system.The authors argue these results support a shift from density regression toward practical dense detection.Source: §6 · p. 11
• LSC-CNN uses a multi-column architecture with top-down modulation to resolve people in dense crowds and places a bounding box on every located person using only point head annotations for training.Source: §6 · p. 11
• Future research could address spurious detections and improve the accuracy of head sizing.Source: §6 · p. 11