Source-linked AI summary
Automatic weld seam segmentation for industrial quality control: a comparison of RGB and polarimetric imaging with CNN and transformer architectures
Simone Garbin, Leonardo Venturoso, Marco Todescato
TL;DR
Visual weld inspection remains labor-intensive and sensitive to acquisition conditions. This study benchmarks RGB and polarimetric imagery with CNNs and transformers, finding that polarimetry reaches 0.93 mask mAP50 while RF-DETR is notably more robust to viewpoint shift than CNNs.
Problem
Evidence remains limited on weld-seam segmentation across controlled and uncontrolled acquisition conditions and under viewpoint shifts, despite inspection’s reliance on variable human judgment.
Method
The study benchmarks CNN and transformer segmentation models on RGB and polarimetric imagery using matched splits, unified evaluation, and multi-seed CNN experiments.
Results
Transformers, especially RF-DETR, retain high accuracy under viewpoint shift, while CNNs collapse; polarimetric imaging reaches a three-seed mean mask mAP50 of 0.93.
Takeaways & Limitations
Small CNNs suffice for fixed viewpoints, whereas RF-DETR is better suited to variable viewpoints and polarimetry can match controlled-RGB accuracy without acquisition control.
Takeaways & Limitations
Small test sets, only three physical RGB specimens, and single-run in-distribution transformer results limit confidence in broader generalization and architectural comparisons.
Abstract
from arXiv · showhide
Visual inspection of welded assemblies remains one of the least automated stages in many industrial production processes, still depending largely on the experience of human operators and thus subject to inter-operator variability; the manufacturing of special-purpose machinery cabins, the setting of this study, is one representative case. This work evaluates the feasibility of automatic weld seam segmentation from RGB and polarimetric imagery, comparing controlled laboratory acquisitions with images captured under real, uncontrolled conditions. Convolutional neural network (CNN) architectures and transformer-based architectures are benchmarked under a unified, threshold-independent protocol, training each CNN with three random seeds to separate genuine effects from seed noise. In controlled RGB conditions, CNN models reach a mean mask mAP50 of up to 0.87, but drop to 0.22-0.48 under uncontrolled acquisition, showing that the acquisition setup is a first-order component of the inspection system. Polarimetric imaging with alignment-preserving geometric augmentation localizes previously unseen welds with a mean mask mAP50 up to 0.93: on par with, rather than ahead of, the best controlled-RGB result, but reaching that accuracy on uncontrolled RGB without requiring acquisition control. The clearest architectural finding concerns viewpoint robustness. In-distribution, transformers and CNNs are broadly comparable; but under a test-time viewpoint shift, the transformer models, and RF-DETR in particular, retain high accuracy while every CNN collapses. The gap holds across three seeds and a resolution-matched control, pointing to architecture rather than training resolution. Within the CNN family, capacity brings no reliable in-distribution gain once seed variance is accounted for: small CNNs suffice for fixed viewpoints, transformers for variable ones.
1. Introduction
Welded-assembly quality control remains poorly automated because visual post-weld inspection is time-consuming, inconsistent, and environmentally sensitive. This study evaluates polarimetric data, CNN and transformer segmentation architectures, and training factors under a unified sample-level protocol.
- Motivation: Visual post-weld inspection remains time-consuming, inconsistent, and sensitive to environmental conditions.The process typically relies on specialized operators.
- Study scope: The study derives six per-pixel maps from polarimetric imagery: Intensity, AoLP, DoLP, Imax, Imin, and Specular.These maps are obtained from the acquired polarimetric data.
- Study scope: Segmentation architectures span single-stage CNNs and transformers across more than an order of magnitude in parameter count.The comparison covers two architectural families with substantially different model sizes.
- Experimental protocol: Experiments assess input resolution, weight initialization, optimizer, and data augmentation under a unified protocol.Data partitions are defined at the level of the physically acquired weld sample.
2. Literature review
The literature spans seam segmentation, architecture selection, polarimetric imaging, small-data learning, and deployment efficiency, but direct evidence for polarimetric weld segmentation remains sparse. Existing studies also lack matched RGB–polarimetric evaluation and rarely separate acquisition effects from architecture choice under shop-floor conditions.
- Research landscape: Literature covers seam segmentation, architecture selection, polarization imaging, small-data learning, and deployment efficiency, with lightweight pipelines dominant in practice.Evidence for polarimetric weld segmentation remains sparse and draws largely on adjacent reflective-surface inspection.
- Inspection methods: Classical inspection methods persist for interpretability and closed-loop integration, while manual inspection remains common because defects, training requirements, and standards vary.Research has shifted toward learned pixel-level seam segmentation, but classical approaches remain relevant in robotic welding.
- Polarimetric imaging: Polarimetric imaging captures geometry, roughness, and material cues largely invisible to RGB, suppressing glare and improving uniformity in reflective-metal inspection.Industrial evidence is strongest in adjacent domains: a polarization system improved YOLOv11 precision by 3.9%, while a polarization-plus-RGB rail-defect network reached 73.00% mIoU.
- Data and deployment: Small, imbalanced datasets constrain industrial deep learning, making transfer learning and augmentation standard mitigations, while deployment studies prioritize efficiency alongside segmentation accuracy.DSGNet achieves competitive surface-defect mIoU with only 0.49M parameters, illustrating the emphasis on compact models for robotic and workstation inspection.
- Research gap: Existing work lacks unified, matched-protocol RGB–polarimetric comparisons and rarely isolates acquisition setup from architecture choice on actual shop-floor data.The literature also seldom evaluates deployment-time viewpoint shifts, motivating controlled analysis of modality, illumination, and architecture under realistic constraints.
3. Materials and methods
The study evaluates weld-seam instance segmentation across controlled RGB, uncontrolled industrial RGB, polarimetric, and close-range imagery using CNN and transformer models. Data splitting, augmentation, training repetition, and common mask-mAP evaluation are designed to prevent leakage and distinguish architectural effects from run-to-run variation.
- Data and annotations: Ground truth consists of manually produced polygonal instance masks for a single Weld class on metallic specimens supplied by an industrial cabin manufacturer.
- Acquisition scenarios: 78 controlled RGB images minimize environmental variability, whereas 54 industrial RGB images capture uncontrolled reflections, clutter, perspective variation, and non-uniform lighting.
- Polarimetric acquisition: 67 physically distinct PolarSens weld scenes yield six 3-channel maps—Intensity, AoLP, DoLP, Imax, Imin, and Specular—with complementary surface information.
- Models and training: The benchmark compares four YOLO-family CNNs with RF-DETR-Seg and Mask2Former transformers, all fine-tuned from COCO-pretrained weights; transformers use reduced resolution because of GPU memory constraints.
- Reproducibility: Each CNN configuration is trained with three random seeds, reporting mean and standard deviation of mask mAP to separate architectural effects from run-to-run randomness.
- Evaluation protocol: Physical-weld-level partitions keep originals and augmentations together, restrict augmentation to training, and evaluate all models on identical splits using COCO-style box and mask mAP50 and mAP50–95.
4. Results
Results show that acquisition conditions strongly affect CNN segmentation, while alignment-preserving PolarSens augmentation enables reliable localization of unseen welds. Under viewpoint shift, transformers retain accuracy where CNNs collapse, whereas CNN capacity provides no reliable in-distribution advantage once seed variance is considered.
- Acquisition conditions: 0.22–0.48 mean test mask mAP50: CNN performance under uncontrolled shop-floor acquisition, versus 0.79–0.82 for the same models under controlled conditions.Three-seed standard deviations are 0.04–0.05, and the best mean CNN is YOLOv11-n at 0.477.
- PolarSens imaging: 0.70 mask mAP50: multi-map PolarSens input without augmentation substantially improves over single-map intensity, whose best mask mAP50 is 0.40.The result indicates that complementary polarization maps carry discriminative information even when processed independently.
- PolarSens imaging: 0.74–0.91 test mask mAP50: all four CNNs reliably localize previously unseen welds using six PolarSens maps with alignment-preserving geometric augmentation.RF-DETR-Seg is essentially tied with the best CNN on mask mAP50, while CNNs retain the advantage on mAP50–95.
- Viewpoint robustness: Under a test-time viewpoint shift, transformer models retain high accuracy while every CNN collapses, with the advantage persisting across three seeds and a resolution-matched control.In-distribution, RF-DETR is competitive or best across scenarios, while Mask2Former is generally weaker except on industrial RGB test.
5. Discussion and conclusions
The discussion identifies acquisition quality as the dominant system factor, while polarimetric imaging matches controlled-RGB accuracy under uncontrolled conditions. Transformers, especially RF-DETR, provide the clearest robustness advantage under viewpoint shift, whereas CNN capacity offers no reliable in-distribution gain.
- Acquisition quality: 0.78–0.87 mask mAP50 in controlled conditions falls to 0.22–0.48 on shop-floor imagery, making acquisition quality the largest performance factor.This gap is wider than architectural differences within either scenario.
- Polarimetric imaging: 0.93 three-seed mean mask mAP50 is reached by YOLOv11-n using multi-map polarimetric imaging with alignment-preserving augmentation.On uncontrolled imagery, polarimetric models reach 0.89–0.93 against 0.25–0.62 for RGB models, matching controlled-RGB accuracy without acquisition control but requiring dedicated hardware.
- Model capacity: CNN capacity gives no reliable in-distribution ranking after accounting for seed variance across PolarSens and both RGB scenarios.With three seeds and 8–15-image test sets, nano, small, and YOLOv8-s cannot generally be separated.
- Viewpoint robustness: 0.842 and 0.810 mask mAP50 are RF-DETR’s three-seed means under zero-shot close-range evaluations, versus CNN means of 0.08–0.17 and 0.04–0.19.The advantage persists across training scenarios, and resolution does not explain it: at 1120 px, YOLOv11-n reaches 0.174 while RF-DETR reaches 0.842.
- Limitations and implications: Small test sets and single-run in-distribution transformer results limit confidence in fine-grained comparisons, supporting deployment-viewpoint testing before defaulting to lightweight CNNs.The fastest model runs at 24 ms/img on a desktop GPU, while polarimetric imaging removes acquisition-control burden at the cost of dedicated camera hardware and polarized illumination.