PaperlayerView exact revision on arXiv

Source-linked AI summary

DERA: Detached Edge-Residual Adaptation for Prohibited item Detection

Yonathan Michael, Mohamad Alansari, Mohammed Bennamoun, Dwarikanath Mahapatra, Andreas Henschel, Naoufel Werghi

arXiv:2609.12411v1cs.CV

TL;DR

X-ray prohibited-item detection is challenged by superposition, weak texture, clutter, and fragmented boundaries. DERA learns detached, instance-contour-based boundary guidance and injects it through residual corrections; it improves BEF-Swin results across the three evaluated datasets.

  • Problem

    Superposition, weak texture, clutter, and fragmented threat boundaries make prohibited-item detection and localization difficult in X-ray imagery.

    Source: §I · p. 1

  • Method

    DERA learns a detached boundary prior from instance-mask contours to gate edge-sensitive features that enter early detector stages through zero-initialized residual adapters.

    Source: §I · p. 2, 4) Boundary-Prior Prediction · p. 3, 5) Edge Gating and Residual Injection · p. 4

  • Results

    DERA improves every BEF-Swin result across the three evaluation datasets, with particularly notable AP75 gains on CLCXray and STCray.

    Source: A. Comparison with Existing Methods · p. 6

  • Takeaways & Limitations

    DERA provides a parameter-efficient adaptation strategy for improving detection under overlap, weak texture, and clutter.

    Source: §VI · p. 9

    Boundary guidance does not fully resolve instance separation, strict-IoU localization, or duplicate detections from multiple queries.

    Source: E. Limitations and Failure Cases · p. 9

Abstract

from arXiv · show

Prohibited-item detection in X-ray imagery remains challenging due to object superposition, weak texture, and material clutter which obscure both semantic appearance and object boundaries. We propose \textbf{DERA}, a \textbf{D}etached \textbf{E}dge-\textbf{R}esidual \textbf{A}daptation framework for prohibited item detection under X-ray imagery. DERA combines hierarchical visual features with a parallel pixel-difference edge pyramid and learns an object-specific boundary prior from training-time contours of the instance masks. The detached prior gates edge-sensitive features, which are injected into the early visual stages through residual heads. This staged design preserves the foundation detector at the start of adaptation, isolates boundary supervision from semantic feature learning, and restricts the final adaptation stage to only \(14.7\)K trainable parameters. Evaluated on PIDray, CLCXray, and STCray, DERA improves the baseline by \textbf{3.1}, \textbf{1.6}, and \textbf{2.4} AP points, respectively.

I. INTRODUCTION

X-ray screening imagery makes prohibited-item localization difficult because overlap, clutter, weak texture, and fragmented boundaries obscure objects. DERA addresses this by learning boundary information separately and applying it through restricted residual corrections.

I. INTRODUCTION

  • X-ray baggage imagery combines superposition, weak texture, blurred boundaries, clutter, and occlusion, with small and thin items especially difficult to detect.Benchmark analyses associate missed detections with reduced object size and show localization worsening as concealment increases.Source: §I · p. 1
  • Benign contours can make indiscriminate edge fusion amplify clutter, while jointly optimized boundary supervision can introduce negative transfer.These concerns motivate learning boundary information separately and restricting its influence to a detection-optimized residual correction.Source: §I · pp. 1–2
  • DERA learns instance-mask-derived contours in a detached branch that gates edge-sensitive features before zero-initialized residual adapters correct early contextual stages.This preserves the foundation detector at the start of adapter optimization and isolates boundary supervision from its feature learning.Source: §I · p. 2
  • DERA combines hierarchical contextual modeling with a multiscale pixel-difference feature pyramid and separates foundation adaptation, boundary learning, and residual tuning.The design targets complementary semantic and geometric representation through controlled multi-stage training.Source: §I · p. 2

A. X-Ray Prohibited-Item Detection

  • Although transformers and vision-language detectors improve global modeling and prompt-conditioned localization, explicit boundary evidence remains underused when object support is weak or fragmented.Prior X-ray work identifies overlap, clutter, and concealment as central localization challenges.Source: §I · p. 1, A. X-Ray Prohibited-Item Detection · p. 2

B. Edge-Aware Representation and Controlled Adaptation

X-ray edge cues can reveal contours where texture is weak, but benign structures make unrestricted edge fusion risky. DERA combines contextual and pixel-difference streams and uses instance-mask-derived boundary guidance to regulate residual corrections.

B. Edge-Aware Representation and Controlled Adaptation

  • Hierarchical side outputs, shape streams, and pixel-difference convolutions provide precedent for keeping edge and semantic processing complementary.PiDiNet uses gradient-sensitive pixel-difference operations for local transition modeling.Source: B. Edge-Aware Representation and… · p. 2
  • Strong X-ray edges may come from benign structures and material transitions, so directly fusing generic edge cues can amplify clutter.Edge information is relevant where threat texture is weak and portions of contours remain visible.Source: B. Edge-Aware Representation and… · p. 2
  • Unrestricted shared optimization can cause negative transfer, motivating isolated contour learning and detached, zero-initialized residual adapters.Prior adaptation approaches use frozen representations, small additive modules, or side networks to separate new learning from the original pathway.Source: B. Edge-Aware Representation and… · p. 3

A. Overview

  • DERA pairs hierarchical transformer context with pixel-difference features, then uses instance-mask-derived contour predictions to gate early edge features and zero-initialized residual projections.Instance masks provide training-time contour targets only; they are not required at inference.Source: A. Overview · p. 3

B. Dual-Stream Foundation Detector

The foundation detector builds contextual and edge-sensitive representations in parallel. It aligns and adds the two streams at each hierarchy level to form the multiscale visual representation used by the detection head.

1) Contextual and pixel-difference features:

  • A hierarchical visual branch produces contextual feature maps while a parallel pixel-difference branch produces an edge-sensitive pyramid.Alternating pixel-difference and ordinary convolutions retains local transition cues alongside conventional appearance information.Source: 1) Contextual and pixel-difference… · p. 3
  • Pixel-difference blocks use depthwise central, angular, or radial difference operators, or ordinary convolution, with residual mapping adapted for downsampling.The outputs collected at four hierarchy levels form the edge-sensitive pyramid.Source: 1) Contextual and pixel-difference… · p. 3

2) Cross-Stream Fusion:

  • At each hierarchy level, stage-specific projections and optional resizing align the streams before elementwise addition forms the multiscale visual representation.The fused pyramid is used by the subsequent detection head.Source: 2) Cross-Stream Fusion · p. 3

3) Detection Head:

The detection head combines fused visual features with category prompts to predict class scores and boxes. A detached boundary prior is learned from high-resolution edge features without sending boundary loss into the feature extractor.

3) Detection Head:

  • The detection head follows the baseline and remains unchanged.Source: 3) Detection Head · p. 3
  • The prompt-conditioned detection head predicts class scores and bounding boxes for each object query.It jointly encodes the fused pyramid with category prompts; the classification branch uses encoded prompt tokens, and the box branch predicts normalized coordinates.Source: 3) Detection Head · p. 3

4) Boundary-Prior Prediction:

  • Two highest-resolution pixel-difference features feed independent side heads, whose aligned logits are concatenated and fused into a boundary prior.The fusion weights are initialized to combine the two scales equally.Source: 4) Boundary-Prior Prediction · p. 3
  • Detaching the edge features and boundary prior prevents boundary loss from modifying the feature extractor and detection optimization from updating the boundary branch.Source: 4) Boundary-Prior Prediction · p. 3

5) Edge Gating and Residual Injection:

DERA gates edge-sensitive features with a detached boundary prior and injects the resulting corrections only into the first two visual hierarchy levels.

5) Edge Gating and Residual Injection:

  • At the first two hierarchy levels, the resized boundary prior is broadcast across the corresponding pixel-difference feature channels.Source: 5) Edge Gating and Residual Injection · p. 3
  • The overview shows visual and edge-branch features fused for prompt-conditioned detection, with contour-guided residual corrections restricted to early visual stages.Source: Figure 2 · p. 4
  • Zero-initialized residual heads initially produce no correction, preserving the foundation features before residual training.Each stage-specific head is a 1×1 projection to the corresponding feature channel dimension.Source: 5) Edge Gating and Residual Injection · p. 4

C. Boundary Supervision

Boundary supervision uses thin inner contours extracted from instance masks, while a three-stage strategy separates detector adaptation, boundary learning, and residual tuning.

C. Boundary Supervision

  • Thin inner contours are extracted from each object mask, merged, resized to boundary-logit resolution, and masked to exclude padded image regions.The boundary branch is optimized using focal loss.Source: C. Boundary Supervision · p. 4
  • The contours are required only for boundary pretraining and are not used during inference.The focal loss uses α = 0.75 and γ = 2, with padded regions excluded from optimization.Source: C. Boundary Supervision · p. 4

D. Training Strategy

  • Training proceeds in three consecutive stages to separate semantic detector adaptation from auxiliary boundary learning.Source: D. Training Strategy · p. 4

1) Foundation Adaptation:

  • During foundation adaptation, the visual and edge branches, projections, and detection modules are jointly adapted to X-ray data while the language encoder remains frozen.Source: 1) Foundation Adaptation · p. 4
  • The visualization depicts edge features preserving object structure under heavy overlap and faint boundaries; green dashed boxes mark ground truth and blue boxes mark DERA predictions.Source: Figure 3 · p. 4

2) Boundary-Head Learning:

The training stages isolate boundary-head learning from detector adaptation, then tune only residual heads against the detection objective. At inference, the frozen boundary branch gates edge features for early-stage corrections without requiring contours or masks.

2) Boundary-Head Learning:

  • Boundary-head learning freezes the foundation components and residual heads, training only the side heads and fusion layer with mask-derived boundary supervision.This prevents the boundary supervision from modifying the established detector.Source: 2) Boundary-Head Learning · p. 4

3) Residual Adaptation:

  • Residual adaptation freezes the trained boundary branch and foundation model, optimizing only zero-initialized residual heads with the detection objective.Detection gradients pass through the frozen network to injection points but update only the residual heads.Source: 3) Residual Adaptation · p. 4

E. Inference

  • At inference, the frozen boundary branch gates edge features for corrections to B0 and B1, while deeper features B2 and B3 remain unchanged.No instance masks, contour targets, or external edge maps are required.Source: E. Inference · pp. 4–5

A. Implementation Details

  • The implementation uses Grounding DINO, a Swin Transformer backbone, and a PiDiNet pixel-difference edge pyramid.The stages are transferred sequentially, and the residual-adapted checkpoint with the best validation performance is evaluated.Source: A. Implementation Details · p. 5

B. Experimental Setup

The experiments use a three-stage training procedure and evaluate class-conditioned bounding-box detection on PIDray, CLCXray, and STCray. The section also reports adaptation and inference costs and defines the AP metrics used for comparison.

B. Experimental Setup

  • Training proceeds through foundation adaptation, boundary-head learning, and residual adaptation, with instance masks needed only to extract training contours.The residual stage freezes the foundation detector and boundary head and updates only zero-initialized residual projections; inference uses an X-ray image and category prompts.Source: B. Experimental Setup · p. 5

C. Computational Efficiency

D. Datasets

  • PIDray, CLCXray, and STCray provide complementary X-ray detection benchmarks with varied occlusion, clutter, concealment, and annotation resources.The experiments use bounding boxes and category labels for detection; pixel-level annotations support auxiliary boundary targets during training.Source: D. Datasets · p. 5

E. Evaluation Metrics

  • AP averages precision over IoU thresholds 0.50–0.95, while AP50 and AP75 use fixed thresholds of 0.50 and 0.75.APS, APM, and APL report AP for small, medium, and large objects, respectively.Source: E. Evaluation Metrics · p. 6

V. EXPERIMENTAL RESULTS AND DISCUSSION

  • All methods use the same dataset partitions, preprocessing, evaluator, and category prompts for vision-language models.Method-specific optimization settings follow their respective implementations.Source: §V · p. 6

A. Comparison with Existing Methods

DERA improves on the baseline across the three evaluation datasets, with a distinct STCray ranking across AP and the fixed-IoU metrics. Scale-wise results and PIDray ablations examine where the gains arise and how adaptation choices affect localization.

A. Comparison with Existing Methods

  • DERA consistently improves on the baseline across all three datasets and improves every BEF-Swin result.The comparison also reports stronger AP75 gains on CLCXray and STCray.Source: A. Comparison with Existing Methods · p. 6
  • On STCray, FR-CNN, CSPCL, and AO-DETR obtain slightly higher AP than DERA, while DERA achieves the strongest AP50 and AP75.The authors note that this ranking is consistent with stronger moderate- and strict-overlap detection but less consistent performance at other IoU thresholds.Source: A. Comparison with Existing Methods · p. 6

B. Performance Across Object Scales

1) Scope of Swin Adaptation:

2) Edge branch study:

  • With the same four-level hierarchy and fusion topology, Sobel performs nearly like baseline in AP (72.2 vs. 72.1) and AP50 (82.8 both), but lowers AP75 from 77.6 to 75.3.The authors say generic gradients provide little additional discriminative information and can degrade precise localization; Sobel responds indiscriminately to material interfaces, overlapping benign objects, and baggage structures.Source: 2) Edge branch study · p. 7

3) Depth of the Boundary-Residual Pathway:

The pathway-depth study favors boundary-guided residual correction at early hierarchy levels, while the initialization comparison favors zero-initialized projections for AP75. Qualitative examples illustrate how boundary priors and localized residual responses accompany improved detections under clutter and overlap.

3) Depth of the Boundary-Residual Pathway:

  • The two-level boundary-residual design performs best with 14.7K trainable parameters, while extending the pathway through P3/S3 raises capacity to 154.1K and reduces performance.Adding P2/S2 provides no gain and slightly reduces AP75; the reported declines at AP75 grow with deeper injection.Source: 3) Depth of the Boundary-Residual… · p. 7, Table VI · p. 7

4) Residual-Projection Initialization:

D. Qualitative Analysis

  • In the shown cluttered examples, DERA recovers targets missed or incompletely detected by the baseline, with residual responses concentrated around spatially relevant structures.The visualizations show fine local transitions at early edge-pyramid levels and progressively coarser patterns at deeper levels.Source: D. Qualitative Analysis · p. 7, Figure 4 · p. 9

E. Limitations and Failure Cases

DERA improves structural representation but does not fully resolve instance assignment, box regression, or duplicate control. The conclusion reports consistent improvements across three datasets and describes DERA as an effective, parameter-efficient adaptation strategy.

E. Limitations and Failure Cases

  • Boundary guidance does not fully resolve instance merging, partial localization, duplicate predictions, or the downstream challenges of instance assignment, box regression, and duplicate control.Figure 5 identifies the representative failure cases.Source: E. Limitations and Failure Cases · p. 9, Figure 5 · p. 9

VI. CONCLUSION

  • Experiments on PIDray, CLCXray, and STCray demonstrate consistent improvements across different imaging conditions.Source: §VI · p. 9
  • DERA is described as an effective, parameter-efficient adaptation strategy for detection under overlap, weak texture, and clutter.Source: §VI · p. 9
Loading 2609.12411v1…