PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Small Patches: Black-Box Detection and Purification of Diverse Backdoor Triggers

Ahmed Abdelnaby, Mohamed Elmahallawy

arXiv:2609.03139v1cs.CVcs.CR

TL;DR

Backdoor defenses may depend on model internals, training data, or clean validation samples, which can be unavailable for black-box deployed models. TRIM localizes prediction-influencing regions and selectively purifies them; experiments report low attack success across diverse attacks, with a caveat for interacting triggers.

  • Problem

    Backdoor attacks can preserve benign-input accuracy while causing malicious predictions on triggered samples, and existing defenses may require unavailable internals, training data, or clean validation samples.

    Source: §1 · pp. 1–2

  • Method

    TRIM segments inputs, filters and selectively reconstructs candidate regions, verifies their influence through black-box prediction changes, and purifies verified regions while caching features.

    Source: §1 · p. 2, §4.3 · pp. 7–8

  • Results

    Across diverse attacks, TRIM reduces attack success rate to as low as 1.16% while maintaining clean accuracy up to 87.87% and outperforming state-of-the-art defenses.

    Source: §1 · p. 2

  • Takeaways & Limitations

    TRIM provides inference-time backdoor mitigation without access to model internals, training data, or clean validation sets.

    Source: §6 · p. 14

    When a dominant trigger independently maintains the backdoored prediction, suppressing a weaker trigger alone may not change the prediction and can leave it undetected.

    Source: §4.3 · p. 8, §5.5 · p. 12

Abstract

from arXiv · show

Deep neural networks (DNNs) are increasingly deployed in real-world vision systems, yet their predictions can be covertly manipulated by backdoor attacks, in which malicious triggers cause targeted misclassification while preserving high clean accuracy. Existing defenses often rely on model internals, training data, or clean validation samples, making them difficult to deploy when only black-box access to a trained model is available. We propose TRIM (Trigger Removal by Identifying Manipulated Regions), a deployment-oriented black-box defense that detects and selectively removes backdoor triggers at inference time without requiring model internals, training data, or clean samples. The key insight behind TRIM is to identify image regions that are responsible for anomalous model behavior and purify only those regions while preserving benign content. TRIM innovates via three key components: (i) region-based segmentation with deep feature representations, (ii) adaptive trigger discovery through inpainting and diffusion-based reconstruction to isolate regions responsible for misclassification---without assumptions about trigger type, shape, or location, and (iii) selective region purification that cleans poisoned regions while retaining benign content. To support practical deployment, TRIM further caches feature embeddings of previously identified triggers, enabling efficient recognition and avoiding redundant detection and purification. Extensive experiments across diverse datasets and backdoor types, including blended, sparse, varying-size, and multiple triggers, show that TRIM consistently outperforms existing black-box defenses, reducing attack success rates (ASR) to as low as 1.16% while preserving clean accuracy of up to 87.87%. These results demonstrate that effective backdoor mitigation is possible at inference time even when the defender has no access to any auxiliary data.

1 Introduction

TRIM targets black-box backdoor defense when triggers vary and conventional defenses may require unavailable model internals, training data, or clean samples. It localizes influential regions for selective purification and reports strong results across diverse attacks.

  • Backdoored models can preserve benign-input accuracy while producing malicious predictions on triggered samples, and modern triggers vary in appearance and location.Source: §1 · p. 1
  • TRIM requires no model internals, training data, clean validation samples, or prior trigger knowledge, and selectively reconstructs and purifies influential regions.Source: §1 · p. 2
  • Caching previously verified benign and suspicious representations reduces redundant model queries during inference-time defense.Source: §1 · p. 2
  • 1.16% ASR and up to 87.87% clean accuracy: TRIM outperforms state-of-the-art defenses across the evaluated diverse attacks.Source: §1 · p. 2

2 Related Work

Prior work addresses backdoors at different model-lifecycle stages, while inference-time approaches are especially relevant when only prediction access is available. TRIM’s distinction is to verify which localized regions influence predictions before selectively purifying them.

  • Backdoor attacks vary in objectives and trigger properties, including spatial coverage, fusion, and sample dependence, motivating defenses without fixed trigger assumptions.Source: §2 · pp. 2–3
  • Pre-training, training-time, and post-training defenses can depend on training data, control of optimization, auxiliary data, model access, or trusted references.Source: §2 · p. 3
  • Inference-time defenses act on incoming samples, but existing approaches often detect, localize, or broadly transform inputs without jointly verifying influential regions.Source: §2 · p. 3
  • TRIM filters and selectively reconstructs candidate regions, checks prediction changes through black-box queries, then purifies influential regions to limit unnecessary changes to benign content.Source: §2 · p. 3

3 Problem Statement, Threat Model, and Defense Assumptions

The threat model considers a model compromised before deployment through poisoned training data, with normal behavior on clean inputs and attacker-controlled behavior on triggered inputs. TRIM’s defender operates in a strict black-box setting using model queries alone.

  • An adversary poisons a subset of training data before deployment, producing a model that behaves normally on clean inputs but exhibits attacker-controlled behavior on triggered inputs.Source: §3 · pp. 3–4
  • The defense-access taxonomy distinguishes white-box access to internals, gray-box partial access, and black-box interaction through input-output queries alone.Source: §3 · p. 4
  • TRIM uses only model predictions at inference time, without training data, model internals, clean validation samples, or prior trigger knowledge.Source: §3 · p. 4

4 Methodology

TRIM uses region-level analysis and black-box prediction feedback to identify suspicious regions, then selectively purifies them while preserving benign content. Its pipeline combines segmentation and feature extraction, reconstructed-region interventions, and verification with caching and mask aggregation.

4 Methodology

  • TRIM requires only black-box model predictions and selectively purifies verified trigger-related regions instead of transforming the entire input.The method separates candidate filtering, region verification, and final purification to limit modification of benign content.Source: §4 · pp. 4–5

4.1 Region-Based Feature Extraction

  • Prompt-free SAM segmentation produces candidate regions that a pretrained encoder represents with normalized features independent of the protected classifier.Candidates can include objects, object parts, textures, or anomalous patterns; features support filtering and comparison with cached regions.Source: §4.1 · pp. 5–6

4.2 Segment-Wise Image Inpainting

  • TRIM suppresses and reconstructs one candidate region at a time, using diffusion inpainting for high-resolution inputs and lightweight Telea inpainting for low-resolution inputs.These intervention images test a region’s influence on the classifier; they are not retained as final purified inputs.Source: §4.2 · pp. 6–7

4.3 Trigger Detection, Isolation, and Purification

  • After excluding large regions, TRIM flags candidates whose suppression changes the prediction, reuses sufficiently similar cached representations, and unions verified masks for one final purification.Caching avoids redundant reconstruction and classifier queries, while final purification is restricted to the union of verified regions.Source: §4.3 · pp. 7–8
  • A region-wise check can miss interacting non-dominant triggers when suppressing either trigger alone does not change the prediction in a previously unseen input.Regions above an empirically selected area threshold are also excluded from verification and purification to limit over-sanitization.Source: §4.3 · p. 8

5 Experiments and Results

Across CIFAR-10 and ImageNet-10, TRIM is evaluated under varied attack objectives, trigger sizes, trigger constructions, poisoning rates, and deployment configurations. The experiments report strong attack suppression alongside clean-accuracy retention, while also identifying limits involving sample-specific triggers and computational cost.

5.1 Experimental Settings

  • Experiments cover CIFAR-10 and ImageNet-10, three attack objectives, eight trigger constructions, and a two-trigger setting, using prediction-only access to the protected classifier.The standard setting poisons 10% of training samples and uses ResNet-50 as the protected classifier.Source: §5.1 · p. 9

5.2 Comparison with Baseline Defenses

  • On CIFAR-10, TRIM reduces average ASR from 97.94% to 3.77% while retaining 85.83% CA; on ImageNet-10, it reduces average ASR from 78.45% to 5.06% while maintaining 76.81% CA.Against FLARE on ImageNet-10 Physical, TRIM achieves 5.18% ASR with 78.37% CA, versus FLARE's 6.37% ASR with 16.33% CA.Source: Table 1 · p. 10, §5.2 · p. 10

5.3 Robustness Across Attack Objectives

  • Across CIFAR-10 attack objectives, TRIM's average ASR is 4.71% for A2O, 4.56% for A2A, and 4.04% for UT; on ImageNet-10, average ASR is 5.76% for A2O and 6.90% for A2A.TRIM maintains CIFAR-10 ASR below 8.1% in every reported attack-mode configuration; ImageNet-10 average CA is 78.06% for A2O and 72.29% for A2A, compared with FLARE's 55.22% and 55.64%.Source: §5.3 · pp. 10–11, Table 2 · p. 11, Table 3 · p. 11

5.4 Robustness to Trigger Size

  • For 7×7 CIFAR-10 triggers, TRIM achieves 1.07% average ASR and 85.49% average CA, compared with 13.06%, 63.93%, and 7.06% ASR for ShrinkPad, ZIP, and FLARE.TRIM records 1.79% ASR with 86.70% CA for Physical and 0.87% ASR with 82.56% CA under LC.Source: §5.4 · p. 11, Table 4 · p. 12

5.5 Generalization to Diverse Trigger Constructions

  • For diverse trigger constructions, TRIM reduces CIFAR-10 ASR from 97.17% to 3.30% for one trigger, from 91.56% to 3.45% for two triggers, and from 88.21% to 2.03% for DBA.On ImageNet-10, VSSC ASR falls from 91.61% to 5.13% with 78.73% CA; input-aware ASR falls from 98.12% to 16.34%, remaining higher than for the other evaluated attacks.Source: Table 5 · p. 12, §5.5 · p. 12

5.6 Sensitivity and Component Analysis

  • Restrictive segment-area thresholds reduce candidate regions and computation but can miss trigger regions and raise residual ASR, while permissive thresholds risk modifying benign semantic content.ASR is minimized around thresholds of 0.2–0.3, and 0.25 is fixed as the default across remaining experiments without attack-specific tuning.Source: §5.6 · pp. 12–13, Table 6 · p. 13
  • Across 5%–20% poisoning, ASR stays below 6%; after 1,000 images, caching reduces regions requiring inpainting to 10.23% on CIFAR-10 and 27.66% on ImageNet-10.The larger remaining fraction on ImageNet-10 is consistent with greater visual diversity and fewer repeated region representations.Source: §5.6 · pp. 13–14

5.7 Computational and Deployment Analysis

  • Stable Diffusion inpainting takes approximately 5.87 s per reconstruction and 4.85 GB of VRAM, so TRIM is not universally real-time with diffusion enabled; lighter modules offer a lower-cost configuration.MobileSAM segmentation takes 0.022 s with 0.08 GB of peak VRAM, while cv2.inpaint takes approximately 0.020 s.Source: Table 8 · p. 14, §5.7 · p. 14

6 Conclusion

TRIM is an inference-time black-box defense that localizes and removes trigger regions without model parameters, training data, or clean validation sets. On CIFAR-10 and ImageNet-10, it substantially reduces ASR while maintaining high clean accuracy.

6 Conclusion

  • TRIM combines semantic segmentation, segment-level features, and selective inpainting to localize and remove trigger regions.The conclusion describes this as restoring correct predictions while preserving access-independent deployment.Source: §6 · p. 14
  • ASR falls from 97.94% to 3.77% on CIFAR-10 and from 78.45% to 5.06% on ImageNet-10.The conclusion reports these reductions while maintaining high clean accuracy.Source: §6 · p. 14
  • TRIM provides stronger protection than ShrinkPad, ZIP, and FLARE against small, large, blended, and spatially sparse triggers.The conclusion reports this comparison across the described diverse trigger patterns.Source: §6 · p. 14

A Evaluation of TRIM on Additional Datasets

Evaluation on additional datasets spans low- and high-resolution images and varied visual domains. TRIM suppresses ASR across the reported datasets and attacks while preserving clean accuracy, including on complex-background and high-resolution settings.

A Evaluation of TRIM on Additional Datasets

  • The evaluation covers CIFAR-100, Caltech-101, and GTSRB across low- and high-resolution images and varied visual domains.GTSRB includes varied illumination, viewpoint, and occlusion; Caltech-101 adds higher resolution and intra-class variability.Source: A Evaluation of TRIM on Additional… · p. 20
  • On GTSRB, ASR drops from 99.21% to 4.22%, and on Caltech-101, from 95.33% to 2.26%.The text reports substantial ASR reductions for these more challenging settings, including physical attacks and higher-variability datasets.Source: A Evaluation of TRIM on Additional… · p. 20
  • For BadNets and Blended attacks, post-defense ASR is reduced to 0.34–6% in most settings.The reported untreated ASR for these classical attacks is typically 95–99%.Source: A Evaluation of TRIM on Additional… · p. 20
  • Clean accuracy remains stable and in some cases slightly improves, reaching 99.01%.The passage presents this as evidence that purification avoids over-purification and preserves task-relevant visual features.Source: A Evaluation of TRIM on Additional… · p. 20

B Sensitivity Analysis

The sensitivity analysis finds that TRIM remains robust across a range of parameter settings, with moderate segmentation ratios and similarity thresholds offering a favorable trade-off. Parameter choices also balance purification coverage, segmentation detail, and runtime.

B Sensitivity Analysis

  • At a segmentation ratio of 0.15, CA is 78.5% while post-defense ASR is 3.6%.Larger ratios process more segments and may degrade clean accuracy through unnecessary modifications.Source: B Sensitivity Analysis · p. 20
  • A higher similarity threshold, such as 0.95, avoids repeated purification of similar regions and improves efficiency while maintaining strong attack mitigation.The minimum SAM mask area also controls retained region size; raising it can reduce segmentation density and runtime, but overly large values may remove useful boundaries.Source: B Sensitivity Analysis · pp. 20–21
  • Moderate mask-merging IoU values, such as 0.5, balance segmentation precision and region compactness.Extreme values may fragment or over-merge masks; fewer SAM sampling points yield coarser, faster masks, while more points increase detail and computation.Source: B Sensitivity Analysis · p. 21
  • Across the tested parameter settings, moderate segmentation ratios and similarity thresholds provide the best trade-off among clean accuracy, attack mitigation, and computational efficiency.The analysis uses an ImageNet-10 subset with 10% poisoning and a 24 × 24 physical patch trigger.Source: B Sensitivity Analysis · pp. 20–21

C Comprehensive Comparison Across Attack Modes for ImageNet-10

On ImageNet-10, TRIM achieves the lowest average ASR across A2One and All2All settings while maintaining clean accuracy comparable to the undefended model. The reported comparison contrasts this robustness–utility trade-off with other defenses.

C Comprehensive Comparison Across Attack Modes for ImageNet-10

  • ShrinkPad and ZIP reduce ASR to moderate levels but have average CA of 71.89% for A2One and 70.19% for All2All.For example, ShrinkPad reports 8.43% ASR for A2One BadNets and 7.65% ASR in All2All.Source: C Comprehensive Comparison Across… · p. 21
  • FLARE suppresses ASR in some cases, including 6.37% for A2One Physical, but its average CA is 55.22% for A2One.The passage characterizes FLARE's behavior as mixed across attack settings.Source: C Comprehensive Comparison Across… · p. 21
  • TRIM achieves average ASR of 5.76% for A2One and 6.90% for All2All, with CA of 78.06% and 72.29%, respectively.The text identifies these as the lowest average ASR values across the evaluated settings, with clean accuracy comparable to the undefended model.Source: C Comprehensive Comparison Across… · p. 22

D Grad-CAM Ablation: Trigger Effects and TRIM’s Response

Grad-CAM ablations show that backdoor triggers shift model attention toward injected patches and that purification restores attention to the semantic object. In multi-trigger cases, TRIM detects the dominant trigger first, while realistic deployment uses one purification pass per image.

D Grad-CAM Ablation: Trigger Effects and TRIM’s Response

  • The GTSRB ablation compares a single BadNets yellow square with a configuration combining that patch and a physical-style noise square.The ImageNet-10 ResNet-50 settings compare a BadNets patch and a physical patch across single- and multi-trigger cases.Source: D Grad-CAM Ablation: Trigger Effects… · pp. 23–24
  • In multi-trigger samples, TRIM consistently detects the BadNets patch first, indicating that detection proceeds from more influential triggers to weaker ones.Realistic deployment performs only one purification pass, removing the first detected trigger.Source: D Grad-CAM Ablation: Trigger Effects… · pp. 23–24
  • Across GTSRB and ImageNet-10, purification restores object-centered attention after triggers shift saliency toward injected patches.Attention returns to the traffic sign in GTSRB and the fish in ImageNet-10.Source: Figure 3 · p. 23, D Grad-CAM Ablation: Trigger Effects… · p. 24

E Ablation Study on TRIM’s Detection and Purification Pipeline

Qualitative ablations show that removing benign regions leaves predictions unchanged, while removing the trigger restores the correct class. This pattern holds for both high-resolution ImageNet-10 and low-resolution GTSRB samples.

E Ablation Study on TRIM’s Detection and Purification Pipeline

  • Removing the yellow-square trigger from the poisoned ImageNet-10 image changes the prediction from “Tench” back to “Cassette Player.”Removing other captured regions, including parts of the cassette, mouse, paper, and monitor base, does not change the predicted class.Source: E Ablation Study on TRIM’s Detection… · pp. 24–25
  • For the poisoned GTSRB sample, only removing the physical-style trigger changes the prediction from “Turn Left” back to “Stop Sign.”Removing alphabetic characters or structural elements of the sign does not affect the prediction.Source: E Ablation Study on TRIM’s Detection… · p. 25
  • Across both datasets, TRIM localizes triggers while avoiding benign or semantically essential regions and restores predictions with minimal visual distortion.The study covers high-resolution ImageNet-10 and low-resolution, highly compressed GTSRB images.Source: E Ablation Study on TRIM’s Detection… · pp. 24–25

F Evaluating TRIM’s Effectiveness Against Modern Typographic Attacks

The typographic-attack evaluation tests whether TRIM can remove adversarial text cues that mislead ViT classifiers and VLMs. The qualitative results show object-consistent predictions after purification across clean, handwritten-label, and digitally composited-label examples.

F Evaluating TRIM’s Effectiveness Against Modern Typographic Attacks

Related papers

Loading 2609.03139v1…