CrowdCC: Jointly Benchmarking Corruption Robustness and Deployment Quantization in Crowd Counting

1. Introduction

• Existing corruption, quantization, and efficient-model evaluations largely treat robustness, numerical precision, and realized deployment cost separately.Crowd counting also requires target-consistent annotations when cropping, viewpoint changes, or occlusion move or remove annotated heads.Source: §1 · p. 1
• CrowdCC evaluates 23 deployment-motivated corruptions at three severity levels across four datasets and 26 models, using FP32, FP16, and post-training INT8 TensorRT engines with Jetson Orin NX throughput and energy measurements.Annotations are updated when a corruption changes geometry or visibility.Source: §1 · pp. 1–2
• Capture-chain corruptions produce the largest degradation, while dense scenes have larger absolute errors and sparse scenes larger proportional degradation.Clean accuracy, corrupted error, and relative degradation yield different model orderings; counting performance also does not fully characterize localization fidelity.Source: §1 · pp. 1–2
• INT8 causes larger model-dependent corruption-response shifts than FP16, without a systematic direction, and does not uniformly improve latency or energy.Realized efficiency depends on compiled model–engine structure.Source: §1 · p. 2

2. Related Work

• Crowd-counting studies address weather, adversarial perturbations, domain generalization, and compression, while a concurrent benchmark covers six corruptions for open-vocabulary object counting.CrowdCC instead studies fixed-category counting across image formation and transmission, transforming annotations when geometry or visibility changes.Source: §2 · p. 2
• Efficiency comparisons commonly use parameters, FLOPs, throughput, or latency, but these measures do not determine how robustness and energy change after reduced-precision compilation.Prior real-engine studies largely consider quantized accuracy or deployment cost separately [1] [2].Source: §2 · p. 2
• CrowdCC evaluates FP32, FP16, and post-training INT8 TensorRT engines while jointly measuring corruption response, throughput, and energy.Corrupted accuracy is assessed using both absolute error and degradation relative to the clean baseline.Source: §2 · p. 2

3.1. Task and corruption suite

• The benchmark applies 23 corruptions at three severity levels to full labelled test splits spanning 2,432 images from four crowd-counting datasets.The evaluated tasks include density estimation, point localization, and direct regression, with final predicted counts assessed.Source: §3.1 · p. 2
• Six corruption families cover environmental conditions, optics and motion, sensor and exposure, encoding and transport, scene clutter, and visibility loss.Seven target-altering corruptions transform annotations; MAE is normalized by the visible-head fraction, with removed-head fractions averaging 13.0%.Source: §3.1 · p. 2

3.2. Corruption generation and severity calibration

• Light, Moderate, and High severity are ordered within each corruption but do not imply equal perceptual strength or counting difficulty across corruption types.Calibration uses acquisition or transmission parameters, image-specific head spacing, real-weather appearance statistics, and review of annotation visibility.Source: §3.2 · pp. 2–3

3.3. Metrics

• Mean-corruption MAE measures absolute corrupted error, while relative degradation measures its increase over the same precision’s clean error.Signed and absolute ΔRD capture the direction and magnitude of the precision interaction, respectively.Source: §3.3 · p. 3
• mCE averages corrupted error relative to a same-dataset, same-precision CSRNet reference across corruptions; lower values are better and CSRNet equals one.Unlike RD, mCE compares corrupted error against a fixed reference.Source: §3.3 · p. 3

4. Experimental Setup

• Table 2 identifies evaluated output types, operations retained above INT8, and four-dataset median INT8 speed and energy ratios relative to FP16.Boldface marks INT8 engines that are slower or consume more energy than FP16.Source: Table 2 · p. 4

4.1. Models and precision settings

• The study evaluates 26 models across CNN, transformer, hybrid, and vision–language architectures, with four output types and parameter counts from 0.13–300.74 M.Models are trained with architecture-specific recipes to recover reported clean accuracy.Source: §4.1 · p. 4
• Every model–dataset configuration uses executed TensorRT FP32, FP16, and post-training INT8 engines; median absolute clean-MAE difference between FP16 and FP32 is 0.048%.FP32 is the deployment baseline, and FP16 is the compiled reduced-precision reference.Source: §4.1 · p. 4

4.2. INT8 calibration

• INT8 quantization is applied to eligible weights and activations without retraining, retaining unsupported or accuracy-sensitive operations in FP16 or FP32.Calibration uses fixed count-stratified clean-training subsets; clipping thresholds are selected on clean test MAE, an optimistic clean-accuracy setting that excludes corrupted images.Source: §4.2 · p. 4

4.3. Evaluation platforms

• Accuracy metrics are computed with A100-specific TensorRT engines, while deployment throughput and energy are measured on Jetson Orin NX 16 GB in 40 W mode.Jetson engines use the same exported ONNX graphs without recalibration or re-export, and cost measurements use the same stratified 100-image subset per configuration.Source: §4.3 · p. 4

5.2. Research questions and findings

• 98.75% mean relative degradation for far-field defocus led corruption risks; cast shadow was lowest at 11.50%.The nine most detrimental mechanisms arose during image formation and transmission; all 26 models had their largest degradation within a capture-chain family, though the hardest family varied by model.Source: §5.2 · pp. 5–6, Figure 3 · p. 5

5.1. Clean accuracy and metric interpretation

• Clean MAE leaders differed from the lowest-mcMAE model on three of four datasets, and no model with the lowest median RD had the lowest corrupted error.INT8 outperformed FP32 clean MAE in 41 of 104 model–dataset comparisons and FP16 in 48; the largest improvement was 8.16 for ZIP-Nano on ShanghaiTech A.Source: §5.2 · p. 5, §5.1 · p. 5, Table 4 · p. 6

5.2. Research questions and findings

• Direct-count variants had the lowest median RD at 14.0% and smallest INT8 interaction at 0.75 percentage points, while density-map models had the lowest median mCE at 0.857.No output representation was uniformly superior: direct-count models performed best across capture-chain families, while localization-oriented models performed best under scene clutter and visibility loss.Source: §5.2 · p. 6, Table 5 · p. 6
• Corruption error reached 32.10 in the densest quartile, with a 6.55× difference; normalized increases were 8.70 and 4.34 percentage points in sparse and dense quartiles, respectively, showing greater sparse proportional degradation.Images were stratified by dataset-specific clean-count quartiles.Source: §5.2 · p. 6, Figure 4 · p. 6
• Counting and localization degradation were correlated but distinct (Spearman ρ = 0.75): object distractors and viewpoint shift harmed localization more, while field-of-view loss and depth-aware haze showed the opposite pattern.At FP32, recall decreased by 0.052 on average versus 0.013 for precision; Figure 5 compares counting and localization degradation across four native-point models.Source: §5.2 · p. 6, Figure 5 · p. 7
• INT8’s median absolute change in relative degradation was 1.91 percentage points versus 0.04 for FP16, with INT8 deviating more in 102 of 104 comparisons.INT8 decreased RD in 60 comparisons and increased it in 44 (p = 0.141); FP16 changes largely stayed within the 0.39-percentage-point 95th-percentile range.Source: §5.2 · p. 7, Figure 6 · p. 7
• INT8 was slower than FP16 in 17 of 104 device measurements, and higher reformat overhead per GMAC was strongly associated with lower median GPU-time speedup across 26 models (Spearman ρ = −0.82).Slowdowns occurred both in models with limited INT8 coverage and those requiring frequent layout or precision conversions.Source: §5.2 · p. 7
• Seven INT8 models were Pareto-undominated in energy–mCE space, but mCE varied by a factor of 2.25 within an equal-cost band.The frontier pairs A100 corrupted error with Jetson energy rather than measuring corrupted accuracy on Jetson; dataset-specific frontiers also varied by dataset and precision.Source: §5.2 · pp. 7–8, Figure 8 · p. 8

6. Discussion

• Precision-dependent frontier membership links robustness and deployment cost, while no single metric captures all aspects of model selection.The frontier plots Orin energy per image against mean-corruption MAE for model–precision configurations.Source: §6 · p. 8, Figure 9 · p. 8
• The benchmark does not establish transfer of model rankings or precision effects to naturally degraded captures, and combined corruptions remain untested.Rain and snow calibration matches selected appearance statistics from real-weather images; combinations such as low light, motion blur, and compression were not evaluated.Source: §6 · p. 8
• Conclusions cover the evaluated models, datasets, TensorRT configurations, and Jetson operating mode; runtime and energy findings require validation elsewhere.Pairing A100 robustness with Jetson cost does not establish cross-platform accuracy equivalence, and engine-structure associations are descriptive rather than causally isolated.Source: §6 · p. 8

7. Conclusion

• Across 26 models and four datasets, clean accuracy does not reliably predict corrupted performance, and INT8 does not uniformly improve robustness or efficiency.Precision changes corrupted-error rankings and robustness–cost frontiers, motivating joint model and precision selection.Source: §7 · p. 8

A.1. Corruption and severity specification

• The 23 corruptions are specified by mechanism, severity-dependent parameters, and annotation policy, with transformed head annotations used when geometry or visibility changes.Some corruptions follow established camera or transmission models, while others are defined from their physical effects and calibrated through the stated procedure.Source: §A.1 · p. 12

A.2. Severity calibration

• Three severity levels were visually calibrated on fixed representative image subsets, with the highest level kept countable and levels ordered only within each corruption.Light, Moderate, and High do not equate perceptual strength or counting difficulty across corruption types.Source: §A.2 · p. 12

A.3. Spacing-relative severities

• For four corruptions, severity scales with median nearest-neighbour head spacing because blur and resolution loss depend on crowd structure.The corresponding pixel magnitudes vary by dataset.Source: §A.3 · p. 12

A.4. Annotation integrity

• Seven corruptions alter the valid counting target by cropping, warping, or occluding annotated heads, so removed-annotation fractions are reported by corruption and severity.Source: §A.4 · p. 12

B.1. Corruption visualizations

• Figure B.1 presents all 23 corruptions at three calibrated severities on one UCF-QNRF image for visual comparison, illustrating appearance only.Source: §B.1 · p. 12

B.2. Severity response

• Far-field defocus mean FP32 relative degradation rises from 17.71% to 181.89% across severity levels, the sharpest increase among the corruptions.All 23 corruptions increase monotonically in aggregate, but this does not imply monotonic behaviour for every image or model.Source: §B.2 · p. 12

B.3. Synthetic and real weather

• Synthetic rain and snow severities fall within the 10th–90th percentile ranges of selected real-weather appearance statistics, but this does not test transfer of robustness rankings or precision effects.The comparison uses labelled real-weather images from the JHU-Crowd++ training split.Source: §B.3 · p. 12

B.4. Corruption redundancy

• Corruptions are represented by FP32 response profiles averaging relative degradation across three severities for each model–dataset pair, then compared using Spearman correlation ρ.The appendix retains correlated corruptions because they can arise at different capture-pipeline stages and call for distinct diagnosis and mitigation.Source: §B.4 · pp. 12–13
• Only one of 253 corruption pairs exceeds ρ = 0.90; the highest association is lens flare with depth-aware haze at ρ = 0.944.Within-family correlations have a higher median than across-family correlations (0.659 versus 0.506), but response similarity does not imply equivalent mechanisms.Source: §B.4 · p. 12

C.1. Reproduction of Published Clean Accuracy

• Clean FP32 results are compared with published performance on full test splits, and each model–dataset checkpoint is selected by lowest clean test MAE among retained checkpoints.Published references are available for 84 of 104 model–dataset combinations; the remaining cases largely involve later datasets.Source: §C.1 · p. 13, §B.4 · p. 13

C.2. INT8 calibration

• INT8 uses post-training calibration on count-stratified clean training images, with activation histograms and per-configuration clipping thresholds selected to minimize clean MAE.Checkpoint and threshold selection use clean test images, making this a per-configuration tuned setting rather than an untouched test protocol; denominator effects alone do not establish the direction of robustness-comparison bias.Source: §C.2 · p. 14

D. Detailed Robustness Evaluation

• Tables D.1–D.4 report clean MAE, mean corrupted MAE, and relative degradation for every model–dataset–precision configuration; Table D.5 gives corruption-level relative-degradation rankings.Relative degradation uses the clean MAE at the corresponding precision.Source: D. Detailed Robustness Evaluation · p. 14

D.1. Output representation

• Output-representation results summarize median relative degradation by corruption family, with the direct-count group containing two Transcrowd variants.Because groups differ in size, architecture, and training procedure, the comparison characterizes the evaluated models.Source: §D.1 · p. 14

D.2. Crowd-size analysis

• Crowd-size analysis reports absolute and population-normalized corruption error by density quartile, including means, medians, and 90th percentiles.Median and 90th-percentile statistics characterize the heavy upper tail from occasional large prediction errors.Source: §D.2 · p. 14

D.3. Localization analysis

• Localization robustness is evaluated for four native-point models using within-dataset matching conventions before aggregation.Raw F1 values are not directly comparable across datasets; results include dataset-level and corruption-family views.Source: §D.3 · p. 14

E. Precision-Dependent Robustness

• Precision sensitivity is measured as INT8-versus-FP32 change in relative degradation, normalized by clean error at each precision to isolate corruption-response changes from clean-accuracy changes.The analysis compares corruption-specific shifts with an FP16 control envelope.Source: E. Precision-Dependent Robustness · p. 14

F. Deployment Performance

• Deployment tables report throughput, runtime decomposition, power, and energy per image for each dataset and model–precision configuration.Figure F.1 accounts for full measured latency and also examines engine characteristics relevant to INT8 performance.Source: F. Deployment Performance · pp. 14–15, §F.1 · p. 16

F.1. Runtime decomposition

• INT8 slowdowns are concentrated in GPU execution, while host preprocessing dominates end-to-end latency for most models.Figure F.1 decomposes latency into preprocessing, GPU execution, transfer, dispatch, and engine residual.Source: §F.1 · p. 15

F.2. Structural diagnostics

• Four models without INT8 throughput improvement occupy four of the six lowest INT8 compute-coverage positions: ZIP-Small 52.3%, FFNet-ConvNeXt-T 57.8%, FFNet-MobileNetV3 50.8%, and MCNN 76.9%, versus a model median above 90%.Table F.5 summarizes compiled-engine layer counts and precision mix; its values describe graph structure rather than device execution.Source: §F.2 · p. 16, Table F.5 · p. 18
• Five indicators describe potential INT8 inefficiency, including reformat density, convolution alignment, unsupported depthwise operations, GPU-time share, and INT8 MAC coverage; they are not a predictive performance model.The criteria include at least 50 reformat layers per GMAC and less than 50% INT8 MAC coverage.Source: §F.2 · p. 17

• All 23 corruptions increase aggregate relative degradation monotonically across severity, with far-field defocus rising from 17.71% to 181.89%.Source: Figure B.2 · p. 20
• Synthetic rain and snow severities fall within the JHU-Crowd++ real-weather inter-decile ranges for the selected appearance statistics.Source: Figure B.3 · p. 21
• FP32 corruption-response profiles have median Spearman correlation 0.52; lens flare and depth-aware haze are the only pair above 0.90, at 0.94.Seven additional pairs fall in the 0.80–0.90 range.Source: Figure B.4 · p. 21
• INT8 robustness shifts are shown as corruption-specific changes in relative degradation across 104 paired evaluations, against an FP16 control band of ±0.99 percentage points.The figure summarizes medians, interquartile ranges, and 10th–90th percentiles.Source: Table E.1 · p. 17, Figure E.1 · p. 21