PaperlayerView exact revision on arXiv

Source-linked AI summary

Automated Weld Seam Recognition and 3D Mapping for Robotic Post Processing Using Photogrammetry and Semantic Segmentation

Augustin Raju, Abilash Madavath, Chandra Yuvesh Aubeeluck, Nicolas Pyschny, Felix Hackelöer, Florian Zwanzig

arXiv:2609.03970v1cs.RO

TL;DR

Large-workpiece scanning can be inefficient, while robotic post-processing requires accurate 3D weld geometry. The paper combines image-based seam segmentation and photogrammetric reconstruction; experimentally, it produced a spatial seam map with millimeter-level localization accuracy, with limitations tied to image conditions and surface texture.

  • Problem

    Large-workpiece high-resolution scanning can be inefficient because it captures data irrelevant to post-processing, despite the need for accurate 3D weld geometry for robotic operations.

    Source: §I · p. 1

  • Method

    The workflow uses multi-view images for photogrammetric reconstruction and SegFormer weld segmentation, then projects detected seam splines onto the reconstructed geometry.

    Source: §II · p. 1, B. Weld Seam Segmentation & Centerline… · p. 2

  • Results

    The projected seam map achieved 4.2 mm RMSE against a manually annotated reference weld line and provided millimeter-level localization accuracy.

    Source: §IV · p. 3

  • Takeaways & Limitations

    The approximate spatial seam map can guide subsequent high-precision inspection or measurement and restrict detailed acquisition to regions of interest.

    Source: §IV · p. 3

    The segmentation model’s limited training dataset makes performance sensitive to lighting, reflections, image quality, and weld appearance, while reconstruction relies on image quality and surface texture.

    Source: §V · p. 3

Abstract

from arXiv · show

Accurate identification of weld seam geometries is essential for automated robotic post processing operations such as grinding, finishing, and inspection. For large workpieces, complete surface scanning using high precision laser scanners or structured light sensors can be time consuming and often generates substantial amount of data that are not relevant. This paper presents an experimental vision based pipeline for the approximate localization of weld seams. This serves as a preliminary stage before high precision measurement. The proposed approach aims to reduce the overall scanning effort and data acquisition efficiency. The proposed method includes capturing images of the workpiece from multiple viewpoints, identifying weld seams from the images using semantic segmentation, reconstructing the workpiece using photogrammetry, and projection of identified weld seams into the reconstructed model.

I. INTRODUCTION

Robotic post-processing requires accurate 3D weld-seam geometry, while full high-resolution scanning of large workpieces can be inefficient. The paper investigates a multi-view image pipeline for approximate localization to guide later high-precision measurements.

I. INTRODUCTION

  • Robotic grinding, finishing, and inspection require accurate weld-seam geometry in 3D space.These operations are often still performed manually or with limited automation.Source: §I · p. 1
  • Laser- and structured-light-based point-cloud or mesh methods can provide accurate geometry but are often suited to small components because of scanning speed and measurement-volume limits.The cited photogrammetry studies were also limited to small measurement volumes,.Source: §I · p. 1
  • For large workpieces, full high-resolution scanning can collect irrelevant data, while low-resolution depth cameras often miss narrow or complex weld geometries.A preliminary localization stage can restrict detailed measurements to regions containing weld seams.Source: §I · p. 1
  • The proposed approach combines multi-view smartphone images, weld detection, and photogrammetric reconstruction to localize seams approximately before targeted high-precision scanning.The workflow combines the resulting information to project 3D weld splines onto the reconstructed object geometry.Source: §I · p. 1, Figure 1 · p. 1

II. METHODOLOGY

The workflow processes a shared set of images in parallel for photogrammetric reconstruction and weld-seam detection, then combines their outputs to map seams onto the reconstructed geometry.

II. METHODOLOGY

  • Photogrammetric reconstruction of multiple images produces a 3D representation of the welded component.The images are overlapping, and the same image set is also used for weld-seam detection.Source: §II · p. 1
  • A second processing branch detects weld seams in the same images and produces 2D weld masks.The two branches provide the inputs for the subsequent 3D seam reconstruction.Source: §II · p. 1
  • Information from both branches is combined to reconstruct 3D weld splines and project them onto the object geometry.The projection provides a spatial representation of the weld seam on the reconstructed object.Source: §II · pp. 1–2

A. Data Acquisition and 3D Reconstruction

The acquisition setup uses overlapping multi-view images of a marked workpiece for photogrammetric reconstruction and scaling.

A. Data Acquisition and 3D Reconstruction

B. Weld Seam Segmentation & Centerline Extraction

Weld seams are segmented in images with a binary SegFormer model, then masks are processed into image-space spline-like centerlines.

B. Weld Seam Segmentation & Centerline Extraction

C. 3D Projection and Multiview Fusion

Image-space weld spline points are transformed into the reconstruction’s global coordinates using depth maps and camera calibration. Observations from multiple images are then merged into a consolidated 3D seam representation.

C. 3D Projection and Multiview Fusion

III. EXPERIMENTAL SETUP

The experiment used a marked, welded steel-tube workpiece and smartphone images captured from accessible sides. The image set supported both photogrammetric reconstruction and weld detection.

III. EXPERIMENTAL SETUP

  • The test workpiece joined two identical rectangular hollow steel tubes with a continuous circumferential butt weld and measured 40 × 10 × 5 cm overall.AprilTag marker sheets were placed beside the workpiece; adjacent markers were 5.2 cm apart center to center.Source: §III · p. 2
  • A smartphone camera acquired 68 images at 1920 × 1080 pixels from all accessible sides of the workpiece.The views were collected to cover both the workpiece geometry and weld seam.Source: §III · p. 2
  • The same image set was used for reconstruction and weld detection; Fig. 2 shows an example with the weld mask in green and centerline in red.Source: §III · p. 2, Figure 2 · p. 2

IV. RESULTS AND DISCUSSION

This proof-of-concept study evaluated a workflow that reconstructs the workpiece and projects detected weld splines onto its geometry. The projected seam had a 4.2 mm RMSE against a manually annotated reference weld line.

IV. RESULTS AND DISCUSSION

  • Evaluation was an initial proof of concept focused on overall workflow feasibility, not rigorous accuracy measurement for each processing stage.Source: §IV · p. 2
  • The SegFormer model segmented weld seams in most captured images, but lighting and reflections contributed to missed detections and some non-weld regions were falsely classified.Source: §IV · p. 2
  • The photogrammetric model had approximately 3 mm dimensional error against the workpiece’s actual outer dimensions, with minor local deviations and surface mismatches.The authors considered these inaccuracies acceptable for approximate localization before high-precision measurement.Source: §IV · p. 2, Figure 3 · p. 3
  • 4.2 mm RMSE against a manually annotated reference weld line quantified projected-weld localization on the 3D model.Fig. 4 shows the projected spline points as red dots.Source: Figure 4 · p. 3, §IV · p. 3
  • Despite reconstruction and segmentation errors, the workflow produced a spatial weld map for approximate localization that can guide later inspection or measurement and restrict detailed acquisition to regions of interest.Source: §IV · p. 3

V. LIMITATIONS & FUTURE WORK

The authors identify constraints in both segmentation and photogrammetric reconstruction, and outline improvements to the training data, 3D path refinement, and robot-orientation estimates.

V. LIMITATIONS & FUTURE WORK

  • The segmentation model’s limited training dataset makes performance sensitive to lighting, reflections, image quality, and weld-appearance variation.Source: §V · p. 3
  • Photogrammetric reconstruction also depends on image quality and surface texture.Source: §V · p. 3
  • Future work will expand training data, strengthen augmentation, refine 3D weld paths, and estimate local surface normals and tool orientations.The longer-term objective is executable robot trajectories for grinding, finishing, and inspection.Source: §V · p. 3

VI. CONCLUSION

The initial pipeline results show that image-based weld detections can be projected into approximate 3D weld paths. The integrated workflow localizes seams on reconstructed geometry as a foundation for future robotic post-processing applications.

  • The experimental pipeline combines semantic segmentation, photogrammetric reconstruction, and 3D mapping to project detected weld regions into a common 3D coordinate system.Source: §VI · p. 3
  • Initial results show a custom-trained SegFormer identifies weld regions and converts detections into approximate 3D weld paths using depth maps and camera poses.Source: §VI · p. 3
  • The workflow connects segmentation, mask-based centerline extraction, depth-assisted projection, and multi-view point fusion to localize seams on reconstructed geometry for future robotic applications.The stated applications include grinding, finishing, and inspection.Source: §VI · p. 3
Loading 2609.03970v1…