Source-linked AI summary
MWIR-4-Plastic: The Identification of Complex End-of-Life Industrial Plastic using Mid-wave Infrared Hyperspectral Imaging and Machine Learning
Elias Arbash, Andréa de Lima Ribeiro, Filipa Simões, Ahmed Jamal Afifi, Aldino Rizaldy, Yuleika Madriz, Samuel Thiele, Sandra Lorenz, Margret Fuchs, Pedram Ghamisi, Paul Scheunders, Richard Gloaguen
TL;DR
The paper addresses refined sorting of shredded black end-of-life vehicle plastics, where existing approaches lack spatially resolved analysis and representative shredded-plastic data. It introduces a co-registered multi-modal hyperspectral dataset and spectral-spatial classification pipeline, finding that advanced deep-learning models and object-level aggregation support accurate complex-plastic classification.
Problem
Existing plastic-sorting studies lack spatially resolved analysis for fast bulk processing and representative datasets of shredded black industrial plastics.
Method
The paper combines co-registered RGB, SWIR, and MWIR imaging, foreground segmentation, spectral processing, pixel-wise classification, and object-level majority voting for shredded black-plastic identification.
Results
The benchmark evaluates classical, machine-learning, and deep-learning methods across multi-modal scenes for classifying four black polymers from end-of-life vehicles.
Takeaways & Limitations
The released dataset and methodologies establish a reproducible benchmark for hyperspectral object analysis of complex dark polymer waste.
Takeaways & Limitations
Reliable deployment beyond the studied scenes requires retraining or fine-tuning on representative new data because spectral signatures also reflect surface characteristics.
Abstract
from arXiv · showhide
The automated sorting of shredded black plastics from end-of-life (EOF) industrial waste presents a significant challenge in recycling facilities, primarily due to the limitations of current sensing and analytical approaches. Existing studies predominantly rely on single-point contact-based mid-infrared spectroscopy or laboratory hyperspectral imaging (HSI) setups, which fail to provide the spatially resolved analysis necessary for fast, bulk processing. Moreover, available datasets are laboratory-controlled and focus on intact rather than shredded plastics, hindering further recycling refinement. Black industrial plastics, in particular, are underrepresented, while most classification pipelines depend on manual region selection and rule-based spectral matching, neglecting spatial information and modern deep learning (DL) methods. To address these gaps, we introduce the first publicly available HSI dataset of shredded black plastics from EOF vehicle, comprising four industrial polymers across 13 co-registered RGB, VNIR, SWIR, and MWIR scenes and their segmentation pipeline. We developed a multi-modal spectral-spatial framework that integrates foreground isolation, pixel-wise classification, and object-level majority voting. By adapting advanced hyperspectral transformers from earth observation and incorporating chemometric band selection, we achieve accurate classification of complex black plastics. The study establishes the first comprehensive benchmark using nine processing methods, including chemometric, machine learning, and DL architectures. To ensure reproducibility, the complete dataset and methodologies are publicly released, establishing a benchmark for a hyperspectral object-analysis pipeline in industrial inspection.
1 Introduction
Industrial black-plastic sorting requires spatially resolved, reproducible methods because existing sensing, datasets, and analysis pipelines are poorly suited to shredded end-of-life streams. The paper addresses these gaps with a public multimodal dataset, automated spectral-spatial processing, and a nine-method benchmark.
- Black plastics are difficult to classify with VNIR and SWIR sensors because carbon-black pigments suppress reflected signals and diagnostic absorption features.
- Existing studies largely use contact spectroscopy or laboratory imaging with manual regions or masks, limiting spatially resolved, fast bulk analysis.
- Available datasets are typically small, laboratory-controlled, and focused on intact objects rather than shredded industrial plastics, especially black engineering polymers from ELV waste.
- Prior MWIR classification commonly relies on rule-based matching, handcrafted descriptors, chemometrics, or conventional machine learning, with modern deep learning less explored.
- The paper introduces the first public dataset of shredded black ELV plastics, with 13 co-registered multimodal scenes, four polymer types, and pixel-wise and object masks.
- It combines automated foreground isolation, pixel-wise classification, object-level voting, chemometric band selection, hyperspectral transformers, and a reproducible benchmark spanning nine methods.
2.1 Dataset: Samples and Acquisition
The dataset comprises industrially processed, shredded ELV plastic particles acquired after metal removal and sorting stages. Because residual misclassified materials can contaminate training data, pixel-level ground truth was established from diagnostic spectral features.
- The study acquired 13 hyperspectral scenes of post-shredding and post-sorting ELV plastic particles processed through conventional industrial separation stages.
- Industrial sorting produced enriched polymer-specific streams, but residual non-polymeric or misclassified particles could introduce noise into machine-learning training data.
- Pixel-level ground truth was therefore established using literature-reported diagnostic features and analytical ranges linked to characteristic polymer molecular vibrations.
2.2 Preprocessing
The preprocessing pipeline calibrates and co-registers multimodal images, isolates foreground objects, normalizes sensor data, selects informative bands, and addresses class imbalance before classification.
- Radiometric and geometric corrections: Calibration converts raw sensor measurements into absolute reflectance and applies radiometric, geometric, and RGB–HSI co-registration corrections.Dark-current and white-reference acquisitions reduce session and environmental variability, while fiducial markers support affine camera alignment.
- Foreground-background segmentation: Foreground-background segmentation masks conveyor and table pixels while generating semantic and instance masks for detected objects.The segmentation uses 166-micron-per-pixel RGB imagery and Meta’s Segment Anything Model.
- Normalization: Standard Normal Variate: SNV normalization is applied independently to each sensor before concatenating spectra, preventing one spectral region from dominating shared features.This compensates for reflectance variation caused by surface properties, illumination, emissivity, and differing VNIR-SWIR and MWIR sensing principles.
- Band selection and concatenation: 143 selected bands combine 107 HSI-SWIR bands with 36 HSI-MWIR bands, reducing the model input from 758 bands.Selection targets informative VNIR-SWIR and MWIR fingerprint regions to reduce dimensionality and computational overhead.
- Train-test split and class balancing: Scene-level splitting assigns five challenging scenes to testing, while weighted cross-entropy compensates for imbalanced class frequencies during training.The test scenes contain 5 mm to 11 mm objects, whereas training scenes contain 5 mm to 30 mm objects; PC has 33,783 training vectors.
2.3 Benchmarking experiments
The benchmark compares similarity-based, classical machine-learning, convolutional, and transformer architectures for classifying four complex polymers. These methods differ in whether they model individual bands, local spectral shapes, or global spectral-spatial dependencies.
- Nine processing methods span spectral similarity, statistical machine learning, convolutional neural networks, and transformer architectures.The benchmark includes SAM, SVM, RF, KNN, LDA, 1D-CNN, ViT, and SpectralFormer variants.
- Machine learning: SVM separates the four plastic classes using hyperplanes and nonlinear kernels over complete spectral signatures.Kernel similarities incorporate absorption depth, band shape, and shoulder position rather than relying on isolated absorption bands.
- Machine learning: Random forest selects reflectance thresholds at individual bands, while KNN assigns classes from nearby training spectra in n-dimensional band space.RF performs implicit feature selection through discriminative splits; KNN assumes same-polymer spectra form local clusters.
- Machine learning: LDA projects spectra onto axes maximizing between-class variance before classification.
- Deep learning: 1D-CNN learns local spectral shapes with sliding filters, whereas ViT relates every spectral position through self-attention.The CNN emphasizes local receptive fields; ViT can model joint patterns across non-adjacent absorption features.
- Deep learning: SpectralFormer groups neighboring bands into tokens and models global dependencies, with patch-wise variants additionally using 9 × 9 spatial neighborhoods.The patch-wise design incorporates spatial-spectral context from neighboring pixels on the same shredded object.
2.4 Post-processing
Post-processing converts vector predictions into scene-level pixel masks and then assigns object-level labels through majority voting within segmented object polygons.
- Pixel-wise classification masks are constructed by assigning each hyperspectral-scene pixel the class predicted for its test-set vector.
- Object-level majority voting aggregates pixel predictions within semantic and instance segmentation polygons.
3 Results
The evaluation shows heterogeneous class performance, with object-level aggregation generally improving over pixel-wise predictions. SpectralFormer-pixel, Random Forest, LDA, and SVM perform strongly, while segmentation choice affects overlapping objects and instance-level results.
- Evaluation: F1-score and Cohen’s Kappa were used to evaluate pixel-wise, semantic-object, and instance-level predictions against pixel-wise ground truth.The three stages correspond to blue, red, and green bars in Figure 4.
- Per-class performance: PA and PP were consistently best recognized, exceeding 0.83 F1 pixel-wise and reaching 0.91–0.98 after semantic or instance aggregation.Their performance was insensitive to the majority-voting type.
- Per-class performance: PC showed variable pixel-wise F1 scores of 0.32–0.78, while semantic-object aggregation produced the largest per-class gain by reducing pixel-level noise.Spatial aggregation was particularly beneficial for PC.
- Per-class performance: Styrene remained the most challenging class, with weak-classifier pixel-wise F1 scores of 0.47–0.52 but best aggregated performance reaching 0.90–0.93.The highest ceiling was reported for SpectralFormer-pixel and Random Forest.
- Model comparison: SF-pixel achieved the highest F1-score across all classes, while RF and SVM exceeded 85% F1 and all models inferred in under 1 ms per pixel on an NVIDIA A100 GPU.RF required 1.3 GB of storage, compared with less than 1 MB for SF-pixel.
- Object-level aggregation: Semantic aggregation generally improved classification over pixel-wise predictions, whereas instance performance remained slightly below semantic performance because zero-shot segmentation over-segmented some objects.Instance masks nevertheless preserve separate predictions for overlapping or touching objects.
4 Discussion
The discussion positions the approach as a multi-modal spectral-spatial framework for shredded black-plastic identification. It combines RGB-based segmentation with advanced spectral models and emphasizes broader spectral coverage, rapid learning, and the need for representative training data.
- Multi-modal framework: The approach combines advanced hyperspectral models for spectral processing with RGB segmentation foundation models for spatial aggregation of tiny shredded particles.The dataset contains particles with longest axes smaller than 3 cm, and the pipeline is adapted to multi-scene industrial data.
- Multi-modal sensing: Combining MWIR and SWIR captures complementary information across a broader spectral range than either modality alone, reducing single-sensor spectral blind spots.This is motivated by the difficulty of discriminating dark plastics in conventional VNIR-SWIR systems.
- Deep learning: Transformer-based and lightweight DL models can learn complex nonlinear mappings between spectra, textures, and material classes from high-dimensional multi-modal data.The discussion links this capability to rapid inference and potential conveyor-speed operation.
- Scope and data requirements: Reliable deployment beyond the studied scenes requires representative sample diversity because spectral signatures encode intrinsic composition and surface characteristics.Retraining or fine-tuning on representative new data is therefore needed for performance outside the examined dataset.
- Adaptability: Maintaining representative, well-annotated multi-modal datasets permits continued DL improvement as new waste streams, additives, and contamination patterns emerge.The stated consequence is an adaptable analytical pipeline for dark polymer wastes.
5 Conclusion
The paper presents a co-registered multi-modal dataset and machine-learning pipeline for shredded black end-of-life-vehicle plastics. SpectralFormer outperformed the other evaluated chemometric and learning methods, while majority voting produced object-level identification and instance masks addressed overlaps.
- Conclusion: The study presents co-registered RGB, SWIR, and MWIR data for black ABS, PA, PC, and PP from end-of-life vehicles.The dataset supports coverage of diverse physicochemical features.
- Conclusion: The pipeline isolates foreground objects with zero-shot vision models, evaluates classical and neural spectral algorithms, and aggregates pixel predictions through majority voting.Instance segmentation is used for overlapping-object scenarios.
- Conclusion: SpectralFormer showed classification superiority over the other evaluated chemometric methods in the presented workflow.The conclusion frames the approach as supporting refined plastic sorting in recycling facilities.