Detecting Backdoors in Object Detection via Pre-NMS Prediction Distribution Shift

Longtian Wang; Zhengyu Zhao; Chenhao Lin; Le Yang; Shiwei Wang; Yuhan Zhi; Xiaofei Xie; Chao Shen

Source-linked AI summary

Detecting Backdoors in Object Detection via Pre-NMS Prediction Distribution Shift

Longtian Wang, Zhengyu Zhao, Chenhao Lin, Le Yang, Shiwei Wang, Yuhan Zhi, Xiaofei Xie, Chao Shen

arXiv:2608.19088v1Aug 19, 2026cs.CVcs.AI

TL;DR

Object detection backdoors can evade existing detectors, particularly in scene-level attacks that affect all objects simultaneously. DistScan detects them by measuring pre-NMS class-distribution shifts on clean inputs, achieving 96.99% average accuracy and outperforming the best applicable baseline by 27.32 percentage points.

Problem
Existing backdoor detectors may rely on trigger inversion or architecture-specific assumptions and fail to generalize reliably to scene-level attacks in safety-critical object detection.
Method
DistScan measures divergence between pre-NMS class predictions aggregated from clean samples and the model’s training class frequencies, without weight access, trigger knowledge, or additional training.
Results
96.99% average detection accuracy, outperforming the best-performing applicable baseline by 27.32 percentage points across two datasets, two architectures, and three scene-level attack scenarios.
Takeaways & Limitations
Pre-NMS prediction distribution shifts provide an effective architecture-agnostic signal for detecting scene-level object-detection backdoors from clean inputs.
Takeaways & Limitations
The evaluation assumes defenders know the training class distribution and can obtain clean samples, while the attacker controls poisoned data and training.

Abstract

from arXiv · show

Object detection models deployed in safety-critical applications remain vulnerable to backdoor attacks that cause targeted misbehaviors when a hidden trigger is present. Existing detection methods either rely on trigger inversion or exploit architecture-specific assumptions, and critically, representative existing methods fail to generalize reliably to scene-level attacks, where a single trigger induces anomalous behavior across all objects in the scene simultaneously. We present DistScan, a backdoor detection framework based on a simple but previously unexploited observation: backdoor injection systematically shifts a model's pre-NMS prediction class distribution away from its training class frequencies, even on clean inputs without any trigger present. DistScan aggregates intermediate class predictions over a clean validation set and flags a model as backdoored if the resulting distribution deviates significantly from the training class frequencies, requiring no model weight access, no trigger knowledge, and no additional training. Extensive experiments on MS-COCO and PASCAL VOC across two architectures and three scene-level attack scenarios demonstrate that DistScan substantially outperforms existing methods, improving average detection accuracy over the best-performing applicable baseline by 27.32 percentage points.

1 Introduction

DistScan detects object-detection backdoors by measuring shifts in pre-NMS prediction class distributions on clean inputs, avoiding trigger knowledge, model-weight access, additional training, and architecture-specific assumptions. Across 288 models, it achieves 96.99% average detection accuracy and exceeds the best-performing applicable baseline by 27.32 percentage points on average.

Motivation: Object detectors in safety-critical applications are vulnerable to backdoor attacks that can alter detection behavior when hidden triggers are present.Detection results directly influence downstream decision-making and system safety.
Limitations of Existing Methods: Existing image-classification methods degrade when transferred to object detection, while detection-specific approaches impose significant limitations.The tasks differ fundamentally in model structure and learning objective.
Key Observation: Backdoor injection shifts pre-NMS prediction class distributions away from training class frequencies even on clean inputs without triggers.Benign models’ pre-NMS predictions closely align with training-data class distributions, reflecting learned class-frequency priors.
Method: DistScan quantifies this shift with Jensen-Shannon divergence using a small clean validation set and training class distribution, without additional training or full dataset access.Its validation construction accounts for differences between one-stage and two-stage detectors.
Experimental Validation: 96.99% average detection accuracy is achieved across YOLOv5 and Faster R-CNN, PASCAL VOC and MS-COCO, and three scene-level attack scenarios.The evaluation comprises 288 object detection models.
Experimental Validation: 27.32 percentage points is DistScan’s average improvement over the best-performing applicable baseline.This comparison is reported across the comprehensive evaluation.

2 Background

Object detection jointly localizes and classifies multiple objects, producing intermediate pre-NMS predictions in both two-stage and one-stage architectures. Backdoor attacks introduce trigger-dependent malicious outputs, while classification defenses often fail to transfer because object detection lacks a single-label assumption.

Object Detection: An object detector produces K predictions, each containing a predicted class, bounding box, and confidence score for a detected object.Formally, the predictions are ˆy = {(ˆc_j, ˆb_j, ˆs_j)} from an input image x.
Detection Paradigms: Two-stage models generate class-agnostic region proposals before classification and refinement, whereas one-stage models directly regress boxes and class scores from dense feature maps.Both paradigms produce candidate predictions that are filtered by non-maximum suppression (NMS).
Backdoor Attacks: A backdoor embeds hidden malicious behavior during training, preserving normal clean-input performance but producing attacker-specified outputs when a designated trigger appears.Data poisoning is the predominant attack vector, with stealthier triggers including imperceptible perturbations, geometric warping, and frequency-domain modifications.
Related Defenses: Existing classification defenses include trigger inversion, meta-classifier, removal, and poisoned-sample detection methods, but their single-label assumption does not transfer to object detection.Object detection’s multi-output nature allows backdoor attacks to manifest in more diverse ways than in classification.

3 Methodology

DistScan detects backdoored object detectors by measuring how their pre-NMS prediction class distributions deviate from training-data class statistics. It constructs an architecture-aligned clean validation set, filters predictions by confidence, and flags models whose Jensen-Shannon divergence exceeds a threshold.

Threat Model: The attacker poisons a small amount of training data while controlling poisoned samples and training, leaving the model architecture unchanged.The intended backdoor preserves normal behavior on clean inputs but produces attacker-chosen outputs when a predefined trigger is present.
Detection Signal: Backdoor training shifts pre-NMS prediction class frequencies away from the training distribution by biasing intermediate predictions toward or away from classes.Pre-NMS predictions encode the class-frequency prior learned from training data, making their class distribution a model-independent detection signal.
Detection Framework: DistScan constructs an architecture-aligned validation set, extracts intermediate pre-NMS class predictions, and compares their distribution with normalized training class counts.The validation design accounts for different input distributions exposed to classification heads in one-stage and other detector paradigms.
Decision Rule: The model is flagged as backdoored when Jensen-Shannon divergence between its normalized pre-NMS distribution and the reference exceeds threshold τ.JS divergence is symmetric and finite when some classes have near-zero probability.
Prediction Aggregation: A confidence threshold δ removes low-confidence predictions before retained predictions are aggregated into class-wise counts.This filtering reduces the influence of uniformly distributed low-confidence outputs.

4 Evaluation · 4.1 Experimental Settings

The evaluation spans two detection benchmarks, two major detector architectures, three attack types, and a 288-model pool. DistScan is assessed with standard detection metrics, controlled thresholds, specified validation settings, and comparisons against ODSCAN and MIA.

4.1.1 Datasets and Models.: Experiments use PASCAL VOC and MS-COCO with YOLOv5 and Faster R-CNN using a ResNet-50 backbone.These models represent one-stage and two-stage detection paradigms, respectively.
4.1.2 Evaluation Metrics.: Evaluation reports True Positive Rate (TPR), False Positive Rate (FPR), Detection Accuracy (Acc), and AUROC.TPR measures correctly identified backdoored models, FPR measures incorrectly flagged benign models, and Acc measures overall correct classification.
4.1.2 Evaluation Metrics.: Each method’s detection threshold τ is chosen to maximize accuracy on the evaluation model pool.This gives every method its best possible operating point, while AUROC evaluates discrimination across thresholds.
4.1.3 Evaluation Model Pool Construction.: The attack pool covers object misclassification, object disappearance, and object insertion, implemented following GMA, BLINDING, and SPONGE, respectively.These attacks target class labels, object visibility, and scene-wide spurious detections.
4.1.3 Evaluation Model Pool Construction.: 288 models comprise 4 settings, 2 datasets, 2 architectures, and 18 models per combination.The four settings are three attack types plus one benign setting.
4.1.4 Implementation Details.: Validation uses random full-image sampling for YOLOv5 and equal-size cropped single-object images for Faster R-CNN, with N = 10 instances per class.The resulting sets contain 200 crops for PASCAL VOC and 800 crops for COCO.
4.1.4 Implementation Details / 4.1.5 Baselines.: Pre-NMS extraction uses confidence threshold δ = 0.0005, while baselines retain their original-paper hyperparameters and include trigger-reconstruction ODSCAN and discrepancy-based MIA.Validation-set construction and δ effects are analyzed separately in Section 4.2.2.

4.2 Results

DistScan consistently detects scene-level backdoors across datasets, architectures, and attack scenarios, substantially outperforming applicable baselines in accuracy and AUROC. Its distributional signal remains interpretable and depends on validation inputs matching the model’s training distribution, with confidence filtering avoiding degenerate prediction distributions.

Detection accuracy: DistScan averages 96.30% accuracy on both VOC with Faster R-CNN and COCO with YOLOv5, while ODSCAN remains at 50% across scene-level settings.MIA averages 75.00% on VOC with Faster R-CNN and is inapplicable to one-stage models.
Detection accuracy: 100% accuracy is achieved on most configurations, with degradation to 91.67% on COCO GMA with YOLOv5 and SPONGE with Faster R-CNN, plus 5.56% FPR on YOLOv5 BLINDING.The authors associate these weaker COCO results with more classes and visually similar class pairs.
AUROC: DistScan reaches AUROC 1.00 on most configurations, with its lowest value at 0.91 on COCO SPONGE with Faster R-CNN.MIA ranges from 0.91 on VOC GMA with Faster R-CNN to 0.00 on COCO SPONGE and 0.40 on COCO GMA with Faster R-CNN.
Validation-set construction: For YOLOv5, Equal Number and Random reach 100.00% on all VOC attack scenarios, while Random exceeds Equal Number on COCO at 96.30% versus 93.52%.Random is adopted as YOLOv5’s default validation construction strategy.
Validation-set construction: For Faster R-CNN, Equal Size performs best, improving SPONGE detection to 97.22% versus 55.56% on VOC, because its inputs match RoI-pooled foreground features.The results reverse the pattern observed for YOLOv5, whose classification head is trained on full-image dense predictions.
Distributional signal and threshold: PCA shows benign distributions clustering near training data while backdoored distributions occupy separate regions, with targeted attacks shifting target-class frequencies and SPONGE or BLINDING causing broader shifts.At δ = 0.0000, Faster R-CNN collapses to 50%; performance progressively degrades above 0.0010, motivating confidence filtering.

5 Conclusion

DistScan detects backdoored object detection models by identifying shifts in intermediate class prediction distributions caused by backdoor training, using clean inputs alone. It measures Jensen-Shannon divergence between predictions from a small constructed validation set and expected training class frequencies.

5 Conclusion: DistScan detects backdoors by measuring how intermediate class prediction distributions deviate from training class frequencies on clean inputs.The method uses Jensen-Shannon divergence between the empirical distribution from a small constructed validation set and the expected training frequencies.