PaperlayerView exact revision on arXiv

Source-linked AI summary

Localize Any Object in X-Ray Security Scans without Human Annotation

Yaqi Cai, Mingxuan Liu, Lorenzo Vaquero, Ning Wang, Nan Pu, Feng Xue, Elisa Ricci, Nicu Sebe

arXiv:2610.07326v1cs.CV

TL;DR

X-ray object localization is difficult to generalize because dense annotations are scarce and RGB foundation models do not reliably transfer across X-ray superposition and weak boundaries. LAO-X adapts SAM2 using physics-guided pseudo-labeled scenes synthesized from unlabeled scans. Across six X-ray benchmarks, it improves category-agnostic localization over foundation-model and self-supervised baselines, with gains up to 23.48% points over SAM3 and RAXO.

  • Problem

    X-ray datasets lack the diverse dense mask annotations needed to train general localizers, and RGB objectness priors are insufficient for objects defined by projected density under physical superposition.

    Source: §1 · p. 2

  • Method

    LAO-X mines salient object masks from unlabeled X-rays, synthesizes Beer–Lambert-consistent scenes with local-to-holistic pseudo-labels, and adapts SAM2 through an occlusion-controlled curriculum.

    Source: §1 · pp. 2–3

  • Results

    Across six test-only X-ray benchmarks, LAO-X outperforms promptable foundation models and self-supervised segmentation baselines, with gains up to 23.48% points over SAM3 and RAXO.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The evaluation supports cross-dataset generalization from unlabeled source scans, including transfer to threat-item benchmarks.

    Source: §1 · p. 3

    On DET-COMPASS's invisible subset, LAO-X achieves 26.69 AP50 versus 63.39 AP50 on the full evaluation set, as weak attenuation can make boundaries difficult to distinguish.

    Source: H Failure Cases and Limitations · p. 17

Abstract

from arXiv · show

Universal object localization in X-ray security inspection is critical for automated threat detection in safety-critical venues. However, unlike everyday RGB images that dominate web-scale visual data, X-ray scans exhibit distinct color patterns, ambiguous boundaries, and compositional structures caused by volumetric superposition. These gaps hinder the direct zero-shot transfer of dense perception foundation models trained on web-scale RGB data. Moreover, annotated X-ray data is scarce and requires expert labeling, limiting both the training of generalizable X-ray native models and the adaptation of RGB foundation models for X-ray data via fine-tuning. Given these challenges, the bright promise of highly generalizable perception models, enabled by data scaling laws in the RGB domain, remains largely out of reach for X-ray inspection. To this end, we introduce LAO-X, a self-supervised adaptation framework that Locates Any Object in X-ray scans using diverse synthesized image--annotation pairs with granularity-aware supervision. LAO-X first designs a saliency-guided X-ray object mining module to separate diverse object instances, which are then used for physics-guided synthesis in the absorbance domain. LAO-X further incorporates an occlusion-controlled curriculum strategy to fine-tune a Segment Anything Model 2 (SAM2) localizer, progressively adapting it to X-ray scans with increasing object counts and overlap levels. Experiments on six X-ray benchmarks show that LAO-X substantially improves category-agnostic localization, achieving 2\% to 23\% mAP gains over SAM2 and X-ray specific baselines in heavily cluttered scenarios, entirely without human-annotated labels.

1 Introduction

X-ray screening needs generalizable object localization, but scarce annotations and the physics of superposition limit both supervised X-ray models and direct transfer from RGB foundation models. LAO-X builds physics-guided pseudo-supervision from unlabeled scans and adapts SAM2 for category-agnostic localization.

1 Introduction

  • X-ray screening safeguards safety-sensitive venues, but growing inspection demand and irregular threats complicate manual review and generalizable detection.The text cites airports and large public events as examples of safety-sensitive venues.Source: §1 · p. 2
  • Expert annotation is costly, while RGB foundation models are unreliable on X-rays because density cues, transparency, and superposition replace visible textures and boundaries.Existing X-ray datasets also lack the category diversity and dense mask supervision needed to train a generalized localizer from human annotations.Source: §1 · p. 2
  • LAO-X mines object-centric masks from unlabeled scans, recomposes objects with Beer–Lambert physics, and generates local-to-holistic pseudo masks without human labels.Holistic masks supervise object-level localization, while over-segmented local masks provide complementary fine-structure supervision.Source: §1 · pp. 2–3
  • An occlusion-controlled curriculum progressively trains granularity-conditioned SAM2 from sparse scenes toward severe superposition.This converts unlabeled scans into structured, physics-aware supervision for category-agnostic localization.Source: §1 · pp. 2–3
  • The setting targets category-agnostic X-ray objectness without category names, text prompts, visual exemplars, or target-domain annotations.Source: §1 · p. 3
  • Across six test-only X-ray benchmarks, LAO-X outperforms promptable foundation models and self-supervised segmentation baselines, with gains up to 23.48% points over SAM3 and RAXO.The reported evaluation covers objects from over 300 categories.Source: §1 · p. 3

2 Related Work

Prior X-ray detection commonly depends on annotated, predefined categories, while open-vocabulary methods still need semantic cues at inference. RGB foundation and self-supervised segmentation methods also face a mismatch with X-ray superposition, motivating physics-grounded localization priors.

2 Related Work

  • Fully supervised X-ray detectors learn predefined categories from expert annotations, while open-vocabulary approaches still require category names or visual exemplars at inference.Existing CLCXray and PIXray benchmarks contain fewer than 20 categories, and supervised methods can generalize poorly to out-of-distribution objects and scenes.Source: §2 · p. 3
  • SAM and its variants provide zero-shot class-agnostic segmentation on natural images, but become unreliable with extreme superposition, weak boundaries, and low-density materials.Domain adaptation can require substantial labeled data, which is impractical for X-ray imagery.Source: §2 · p. 3
  • Unsupervised segmentation methods use object-aligned features and hierarchical pseudo-labels, but their RGB-style appearance grouping poorly fits X-ray attenuation and pixel overlap.LAO-X instead mines salient X-ray objects and synthesizes superposed training samples with a physics-guided generator.Source: §2 · p. 4

3 Method

LAO-X turns unlabeled X-ray scans into physics-aware synthetic supervision through saliency-guided object mining, then fine-tunes SAM2 with granularity-conditioned labels and an occlusion-controlled curriculum. The pipeline progresses from sparse objects to severe superposition to support category-agnostic localization without manual annotations.

3.1 Saliency-guided X-ray Object Mining

  • SXOM uses saliency scoring and dual-gated masking to mine foreground objects of varied densities and structures from unlabeled X-ray scans.The mining stage derives object-centric samples for an X-ray item bank used in later synthesis.Source: §3.1 · pp. 4–5

3.2 Physics-guided X-ray Inspection Synthesis and Labeling

  • PXISL places mined objects on real backgrounds and combines their density maps additively in absorbance space to synthesize cluttered X-ray scans.It pairs synthesized scans with instance masks and holistic–part pseudo-labels for object-level and complementary fine-structure supervision.Source: §3.2 · pp. 5–6
  • The synthesis controls scene complexity through object count and overlap, while part masks complement holistic labels for Matryoshka-like overlaps.Each holistic instance receives granularity g=1; part masks receive g<1.Source: §3.2 · p. 6

3.3 Granularity Supervision with Occlusion-Controllable Training

  • LAO-X fine-tunes SAM2 on synthesized scans with holistic–part labels and an occlusion-controlled curriculum, progressing from single objects to severe overlap.The curriculum controls scene difficulty through item count and overlap, while a granularity encoder lets SAM2 predict masks at requested granularity.Source: §3.3 · p. 6

4 Experiments

LAO-X is evaluated on six X-ray benchmarks, including test-only datasets, against promptable segmentation and self-supervised baselines. It leads the reported localization comparisons, while ablations examine its curriculum, granularity behavior, and saliency-based mining.

4.1 Implementation Details

  • The adaptation uses 18,000 physics-guided synthetic scenes and fine-tunes a pretrained SAM2-small using LoRA while freezing the other parameters.COMPASS-XP images supply mining and synthesis data; all six evaluation datasets are test-only, and box localization is assessed with COCO metrics.Source: §4.1 · pp. 6–7

4.2 Comparison with the State of the Art

  • On exhaustively annotated DET-COMPASS, LAO-X achieves the best AP50, AP75, and mAP (63.39%, 55.73%, 53.33%), substantially outperforming all baselines; on five prohibited-item datasets, it consistently surpasses promptable segmentation baselines in AP95.Source: §4.2 · p. 7
  • On PIXray and PIDray, LAO-X achieves the best mIoU and F1; on PIXray, it reaches 64.06%/71.86%, improving over the strongest baseline by more than 10%.Source: §4.2 · p. 8
  • In qualitative comparisons, LAO-X produces object-aligned masks that separate overlapping items more consistently than SAM2, SAM3, and UnSAMv2.Figure 3 presents top-five confident masks across real-world X-ray datasets.Source: §4.2 · p. 8, Figure 3 · p. 8

4.3 Ablation Study

  • Curriculum training raises mAP from 16.49% with mixed synthetic scenes to 18.33% with the Easy → Hard → Extreme schedule.The reported stage-wise progression raises mAP from 15.58% to 16.83% and then 18.33%; progressively increasing occlusion also improves recall under X-ray superposition.Source: §4.3 · pp. 8–9, Table 4 · p. 8
  • With the same point prompt, LAO-X responds to granularity g more consistently than UnSAMv2, moving from part-level to holistic masks as g increases.The comparison is qualitative and uses green predicted-mask overlays under heavy overlap.Source: Figure 4 · p. 9, §4.3 · p. 9
  • Saliency mining highlights foreground items across varied scenes, and LAO-X produces the strongest mined-mask mAP and AR among the compared methods.The object bank is constructed from COMPASS-XP, selected for its richer categories and dual-energy scans.Source: §4.3 · p. 9

5 Conclusion

LAO-X turns unlabeled X-ray scans into structured supervision through saliency-guided mining, physics-guided synthesis, and occlusion-controlled SAM2 training. Across six benchmarks it improves localization, while scanner-specific pseudo-color mapping remains a limitation.

5 Conclusion

  • LAO-X mines object-centric items, synthesizes scans, and trains granularity-conditioned SAM2 with an occlusion-controllable curriculum.The pipeline supports localization of whole objects and parts under severe superposition without manual annotations.Source: §5 · p. 10
  • Across six X-ray benchmarks, LAO-X consistently improves precision, recall, and mask quality over RGB foundation models and self-supervised baselines.The conclusion presents LAO-X as a practical baseline for universal X-ray perception.Source: §5 · p. 10
  • Scanner-specific pseudo-color mapping leaves a domain gap; future work could use a color-mapping sampling model to improve synthesis robustness across scanners.Source: §5 · p. 10

Overview

The appendix covers implementation, broader impact, and evaluation protocols, then adds analyses of segmentation, pseudo-label granularity, synthetic data, interactive prediction, and limitations.

Overview

  • It describes the partial-annotation evaluation protocol and sensitivity analysis, plus segmentation evaluation on a real X-ray inspection dataset.Source: Overview · p. 13
  • Further topics include holistic and partial pseudo-labels, synthetic-scene complexity and data scale, qualitative mask comparisons, multi-granularity interactive prediction, and failure cases and limitations.Source: Overview · p. 13
  • The appendix includes a broader-impact discussion and implementation details.Source: Overview · p. 13

A More on Implementation Details

The appendix details promptable segmentation baselines, X-ray synthesis choices, and granularity ablations, and compares curriculum training cost with single-stage training. The curriculum adds modest measured training overhead, while the comparison excludes object-bank construction and offline synthesis.

Implementation Details of Promptable Segmentation Baselines

  • SAM2 and SAM3 generate masks from internal point grids without ground-truth prompts; thresholds are validation-calibrated, and outputs are converted to tight boxes for box evaluation.Candidate masks are filtered by predicted quality, stability, binarization threshold, and non-maximum suppression; SAM2 and SAM3 use different interfaces and point-grid settings.Source: A More on Implementation Details · p. 13

Dual-energy X-ray imaging

  • Dual-energy synthesis adds attenuation independently in high- and low-energy absorbance domains, preserving material-dependent cues when those channels are available.Because dual-energy access is scanner-dependent, single-energy scans use the same absorbance addition on the available attenuation map.Source: A More on Implementation Details · pp. 13–14

Implementation Details

Ablation study of pseudo-labeling granularity

  • The granularity ablation compares holistic instance masks, over-segmented local parts, and the unified LAO-X strategy across five X-ray benchmarks.Source: Table 6 · p. 14

Computational Cost and Curriculum Overhead

  • 9.64 vs 8.94 hours total; the curriculum adds 0.70 hours (~7.8%), while startup-adjusted training is 8.15 vs 8.46 hours and peak memory is 22 GB in both settings.The comparison uses approximately matched update budgets and reports model-training cost only.Source: A More on Implementation Details · p. 14, Table 5 · p. 14
  • The measured training comparison excludes object-bank construction and offline scene synthesis, so its overhead figures do not cover those pipeline stages.Source: A More on Implementation Details · p. 14

B Partial-Annotation Evaluation Protocol and Sensitivity Analysis

The sensitivity analysis evaluates a region-filtered protocol for partial annotations against full-image evaluation and several expansion factors. LAO-X remains the leading evaluated segmenter, although measured margins depend on the protocol.

Protocol definition

  • The region-filtered protocol retains predicted boxes whose centers fall inside ground-truth boxes expanded by α = 1.5; it is not unrestricted full-image evaluation.Predictions outside retained regions do not contribute to reported metrics.Source: B Partial-Annotation Evaluation… · p. 14, Table 7 · p. 15

Sensitivity to region filtering

  • Relative to the default protocol, the complete ordering of the four methods is preserved in 15 of 16 dataset–metric–protocol combinations, although protocol changes absolute scores and improvement margins.On CLCXray, LAO-X exceeds the strongest baseline by 4.91 AP95.Source: B Partial-Annotation Evaluation… · p. 14
  • On CLCXray, LAO-X exceeds the strongest baseline by 4.91 AP95 50 points under full-image evaluation and 6.23 points under the default 1.5× filter.Source: B Partial-Annotation Evaluation… · pp. 14–15

Unmatched predictions and false discoveries

  • At 70.42% FDR on DET-COMPASS, LAO-X recalls 85.11% of annotated objects versus 54.37% for SAM2; partial annotations still limit assessment of full-image errors.On CLCXray, the default filter removes 70.8%–76.2% of unmatched predictions, which may include correctly localized unlabeled objects as well as errors.Source: B Partial-Annotation Evaluation… · p. 15

C Segmentation evaluation on real X-ray inspection dataset

LAO-X achieves strong segmentation results on PIXray and PIDray, while combining holistic and part-level pseudo-labels improves localization across benchmarks. The results also show gains in object coverage under clutter.

C Segmentation evaluation on real X-ray inspection dataset

D Pseudo label of holistic/partial objects and their combination

  • LAO-X reaches 23.90% AP50 on CLCXray, above Holistic (17.46%) and Part (18.72%), and 26.30% on HiXray versus 16.73% for the compared pseudo-label setting.The combined strategy draws on holistic and part-level annotations, which provide complementary information for localization.Source: D Pseudo label of holistic/partial… · pp. 15–16

E Synthetic Scene Complexity and Data Scale

The curriculum stages synthetic scenes from simpler to more cluttered, and performance is more sensitive to that staged transition than to specific extreme-stage settings. Reducing synthetic data under a fixed-epoch schedule lowers localization metrics and object coverage, while qualitative comparisons favor LAO-X masks.

E Synthetic Scene Complexity and Data Scale

  • Performance remains stable as extreme-stage object count rises to 15 or overlap ratio γ reaches 0.9, while staged curriculum design matters more than specific stage hyper-parameters.The curriculum proceeds sequentially through Easy, Hard, and Extreme stages with increasing object counts and overlap.Source: E Synthetic Scene Complexity and Data… · p. 16
  • Reducing synthetic scenes from 18,000 to 4,500 lowers CLCXray AP50 from 37.61 to 34.19 and AR10 from 50.20 to 40.27 under a fixed-epoch schedule.The reduced dataset also yields fewer optimization updates, so the experiment does not isolate data diversity at fixed computational cost.Source: E Synthetic Scene Complexity and Data… · pp. 16–17

F Qualitative Comparison of Mask Localization Performance.

  • LAO-X produces more object-aligned masks with cleaner boundaries and better instance separation than SAM2, SAM3, and UnSAMv2 in the qualitative comparisons.The visualizations include recovery of plausible objects beyond manually annotated prohibited items.Source: F Qualitative Comparison of Mask… · p. 17

G Quantitative and Qualitative Analysis of Interactive Multi-Granularity Segmentation.

Granularity controls the scale of LAO-X's predicted masks, with the best setting varying by dataset. Across queried values, LAO-X changes mask scale more consistently than UnSAMv2, including in cluttered scans.

G Quantitative and Qualitative Analysis of Interactive Multi-Granularity Segmentation.

  • AP50 rises almost monotonically with granularity on DET-COMPASS, while the best setting varies on prohibited-item datasets: PIDray favors about 0.8 and PIXray peaks locally at 0.3 and highest at 1.0.Granularity is reported as a practical control for adapting output scale to dataset and labeling regime.Source: G Quantitative and Qualitative… · p. 17
  • With the same point prompt, LAO-X moves from compact part-level masks at smaller g toward holistic masks at larger g, while UnSAMv2 can shift masks incorrectly.The reported contrast holds under clutter and heavy overlap.Source: G Quantitative and Qualitative… · p. 17

H Failure Cases and Limitations

LAO-X is challenged by weak attenuation and extreme overlap, and its absorbance-domain synthesis omits several scanner and imaging effects. The paper frames deployment as assistive, requiring target-environment validation and human oversight.

H Failure Cases and Limitations

  • On DET-COMPASS, LAO-X reaches 26.69 AP50 on the invisible subset versus 63.39 AP50 on the full set, with weak attenuation making object boundaries difficult to distinguish.Thin plastic, fabric, or paper may be fragmented or missed when their attenuation cues approach the background.Source: H Failure Cases and Limitations · p. 17, Table 11 · p. 18
  • On PIDray Hidden, LAO-X has lower AR10 (14.60) than SAM3 (31.35) and UnSAMv2 (19.59), despite leading in box AP and AR1.Errors include merging neighboring objects, predicting only a component, or missing instances dominated by overlap.Source: H Failure Cases and Limitations · p. 17, Table 12 · p. 18
  • PXISL does not explicitly simulate scattering, beam hardening, detector noise, calibration differences, or scanner-specific pseudo-color mappings, so evaluated-dataset performance does not guarantee robustness to every scanner or condition.The paper recommends validation and calibration on the target scanner, human oversight, and monitoring of missed objects and other failures.Source: H Failure Cases and Limitations · p. 18

I Broader Impact

  • The paper recommends using LAO-X as an assistive tool rather than the sole basis for security decisions, with human oversight and validation in the target environment.The stated concerns include false negatives, false positives, scanner differences, dataset bias, and changing operating conditions.Source: I Broader Impact · p. 18

Related papers

Loading 2610.07326v1…