Source-linked AI summary
SIXray : A Large-scale Security Inspection X-ray Benchmark for Prohibited Item Discovery in Overlapping Images
Caijing Miao, Lingxi Xie, Fang Wan, Chi Su, Hongye Liu, Jianbin Jiao, Qixiang Ye
TL;DR
Prohibited-item discovery in security X-ray images is difficult because overlapping luggage contents, clutter, and severe class imbalance challenge recognition. The paper introduces the SIXray benchmark and CHR, which combines hierarchical refinement with class-balanced learning. CHR outperforms baselines across classification and localization settings, with stronger gains on less balanced data.
Problem
Security X-ray images contain overlapping objects, complex clutter, and very few positive samples, creating a challenging prohibited-item recognition problem.
Method
The paper builds SIXray and proposes CHR, which uses high-level cues to refine mid-level features and adds a class-balanced loss.
Results
CHR produces consistent gains over baselines in classification and localization accuracy, with more significant improvement on less balanced training data.
Takeaways & Limitations
SIXray establishes a large benchmark and CHR establishes a strong baseline for prohibited-item discovery in overlapping X-ray images.
Takeaways & Limitations
The overlapping-image formulation ignores object relationships and stacking order and is acknowledged as an inaccurate approximation.
Abstract
from arXiv · showhide
In this paper, we present a large-scale dataset and establish a baseline for prohibited item discovery in Security Inspection X-ray images. Our dataset, named SIXray, consists of 1,059,231 X-ray images, in which 6 classes of 8,929 prohibited items are manually annotated. It raises a brand new challenge of overlapping image data, meanwhile shares the same properties with existing datasets, including complex yet meaningless contexts and class imbalance. We propose an approach named class-balanced hierarchical refinement (CHR) to deal with these difficulties. CHR assumes that each input image is sampled from a mixture distribution, and that deep networks require an iterative process to infer image contents accurately. To accelerate, we insert reversed connections to different network backbones, delivering high-level visual cues to assist mid-level features. In addition, a class-balanced loss function is designed to maximally alleviate the noise introduced by easy negative samples. We evaluate CHR on SIXray with different ratios of positive/negative samples. Compared to the baselines, CHR enjoys a better ability of discriminating objects especially using mid-level features, which offers the possibility of using a weakly-supervised approach towards accurate object localization. In particular, the advantage of CHR is more significant in the scenarios with fewer positive training samples, which demonstrates its potential application in real-world security inspection.
1. Introduction
SIXray addresses prohibited-item discovery in X-ray images, where overlapping objects, clutter, and severe class imbalance complicate recognition. The paper introduces a large benchmark and CHR, which combines hierarchical visual cues with class-balanced learning.
- Motivation: Security inspection requires fast and accurate prohibited-item recognition because baggage objects are randomly stacked, heavily overlapped, and mixed with meaningless clutter.These conditions can make prohibited items difficult to distinguish even for human inspectors.
- Dataset and benchmark: SIXray contains more than one million X-ray images and is 100 times larger than the existing largest prohibited-item image collection.Less than 1% of images have positive labels, reflecting the rarity of prohibited items in realistic inspection settings.
- Problem formulation: The paper models an X-ray image as overlapping transparent sub-images sampled from a mixture distribution rather than as a conventional single-class image.The formulation is used to represent multi-class labels arising from overlapping objects.
- Method: CHR uses reversed network connections so high-level semantics assist mid-level features while irrelevant information is filtered through hierarchical refinement.A class-balancing loss is added to address imbalance, and the approach is evaluated across different positive/negative sample ratios.
- Evaluation: CHR achieves higher classification performance than multiple baselines and shows larger improvements when training data are less balanced.The study also provides testing-set bounding boxes and applies CAM as a baseline for weakly supervised localization.
2. Related Work
Prior X-ray recognition datasets and methods do not fully represent the overlapping, cluttered security-inspection setting. The paper positions SIXray and its recognition approach within broader object-recognition and localization research.
- X-ray imaging: X-ray images use pseudo-colors based on material absorption, while baggage scans can show heavily occluded objects because X-rays penetrate the luggage.This makes object overlapping a central distinction from many natural-image settings.
- X-ray benchmarks: GDXray includes prohibited-item categories but has few background clutters, limited overlaps, and relatively few negative samples, making recognition easier.The paper presents SIXray to address these dataset limitations.
- Object recognition and localization: Deep learning methods learn hierarchical visual patterns, with image-level classification and object-level localization representing two common recognition strategies.The paper studies image-level recognition because per-object training annotations are unavailable.
- Object recognition and localization: Weakly supervised localization and class activation methods provide related approaches for localizing objects without full per-object training supervision.Multiple-instance-learning methods instead treat convolutional filters as detectors that activate regions of interest.
- X-ray recognition methods: Earlier X-ray research explored handcrafted features and deep models for compact visual representation or adaptation to X-ray images.The paper distinguishes its setting through overlapping objects and large-scale benchmark construction.
3. The SIXray Benchmark
The SIXray benchmark contains over one million real-world subway-station X-ray images with six prohibited-item categories and strong positive-negative imbalance. Its properties include overlapping luggage contents, substantial appearance variation, clutter, and bias toward negative predictions.
- Dataset construction: SIXray contains 1,059,231 X-ray images collected from several subway stations, with six prohibited-item categories: gun, knife, wrench, pliers, scissors, and hammer.The hammer class has only 60 samples and is excluded from experiments.
- Dataset construction: The dataset preserves real-world imbalance, with far fewer positive than negative images, and its subsets use negative-to-positive ratios of 10×, 100×, and 1000×.SIXray100 is described as closely matching the real-world distribution.
- Annotations and evaluation: Image-level presence labels are available across the dataset, while testing images additionally receive bounding boxes for evaluating object localization.The approach is trained without per-object annotations but evaluated at both image and object levels.
- Dataset properties: Luggage objects are randomly stacked, yet X-ray penetration reveals occluded items, producing the benchmark’s defining overlapping-image challenge.The paper contrasts this with GDXray, which often contains only one item per image.
- Dataset properties: Prohibited items vary in scale, viewpoint, style, and subtype, while open-set clutter introduces uncertain background objects and recognition noise.These factors create substantial intra-class variation and complicate interpretation of background regions.
4. Our Approach
CHR models overlapping X-ray images as mixtures, then uses hierarchical refinement to pass high-level supervision backward through mid-level features. A class-balanced loss further filters irrelevant negative samples, while efficient one-pass optimization avoids iterative training.
- 4.1. Motivation and Formulation: SIXray images are modeled as mixtures of class-specific sub-images under the penetration assumption.The formulation is approximate because it ignores object-overlap relationships and stacking order.
- 4.1. Motivation and Formulation: CHR uses reversed connections so higher-level predictions guide the removal of irrelevant information from mid-level features.Neighboring feature layers receive supervision hierarchically, with the top feature layer connected directly to the label through a classifier.
- 4.2. Approximation with Hierarchical Refinement: Hierarchical refinement iteratively propagates supervision from high-level semantics through intermediate feature layers.The original recurrent optimization is replaced during training by stochastic gradient descent, with each case processed once to improve efficiency.
- 4.3. Class-Balanced Loss: Predictions from all hierarchical stages are averaged into the final output, while CHR can use three complementary feature layers.The implementation applies classifiers to each refined feature and averages their predictions; three layers were found sufficient in practice.
- 4.3. Class-Balanced Loss: The class-balanced loss downweights mid-level negative samples using high-level cues to reduce bias from abundant negative examples.This targets the severe imbalance in prohibited-item discovery, where positive training samples are much fewer than negative ones.
5. Experiments
Experiments evaluate CHR for classification, localization, imbalance handling, ablations, computational cost, and generalization across SIXray subsets and ILSVRC2012. CHR consistently improves classification and localization, with larger gains under stronger imbalance and modest extra computation.
- Classification: Quantitative Results: CHR achieves consistent mean Average Precision gains across all five backbones and the SIXray10, SIXray100, and SIXray1000 subsets.Overall classification results are averaged over five classes.
- Classification: Quantitative Results: 8.22% and 9.08% absolute improvement over SIXray1000 is observed with Inception-v3 and DenseNet, respectively.The paper reports that CHR works better in deeper networks.
- Classification: Quantitative Results: 30% accuracy improvement is reported for scissors with DenseNet, although gun classification is not boosted in every subset.Scissors has the fewest training samples among the five evaluated prohibited-item classes.
- Classification: Quantitative Results: Accuracy gains increase as the negative-positive ratio rises across SIXray10, SIXray100, and SIXray1000.The respective negative-positive ratios are 10, 100, and 1000.
- Localization: Quantitative Results: 5.61% and 9.26% localization improvements over DenseNet are reported on SIXray100 and SIXray1000, respectively.The corresponding accuracies are 50.31% versus 44.70% and 43.87% versus 34.61%.
- Localization: Quantitative Results: 16.04% localization improvement is reported for Wrench on SIXray1000 with Inception-v3+CHR over Inception-v3.The corresponding accuracies are 23.53% versus 7.49%.
- Ablation Studies: 2.37% and 7.51% improvements in classification and localization occur on SIXray100 when hierarchical refinement and class-balanced loss are combined with ResNet34.On SIXray1000, the corresponding improvements are 5.48% and 5.11% over baseline ResNet34.
- Ablation Studies: CHR adds 7.81% processing time on ResNet34, increasing per-image latency from 7.68ms to 8.28ms on a Tesla V100 GPU.The paper characterizes this as a relatively small amount of extra computation.
6. Conclusions
The paper introduces SIXray and CHR for prohibited-item discovery in complex, imbalanced X-ray images. CHR refines mid-level features hierarchically and achieves consistent classification and localization gains, while the overlap formulation remains an identified limitation.
- SIXray provides a large-scale benchmark of more than one million real-world X-ray images with six types and more than 20,000 manually annotated prohibited items.The dataset is designed to facilitate research on a relatively understudied computer-vision application.
- CHR formulates X-ray images as overlaps of several sub-images and refines mid-level features hierarchically and iteratively.The method uses an approximate but efficient non-iterative optimization in practice and adds a loss function for heavy positive-negative imbalance.
- CHR produces consistent gains in both classification and localization accuracy across several popular network backbones, establishing a strong baseline.
- The overlap formulation based on the penetration assumption is inaccurate in many aspects, motivating better physical models and clearer connections to natural-image occlusion.