PaperlayerView exact revision on arXiv

Source-linked AI summary

Adaptive Depth-Map-Guided Bundle Adjustment for Correspondence-Free Multi-View Point Cloud Registration

Yiran Zhou, Yingyu Wang, Shoudong Huang, Liang Zhao

arXiv:2609.01089v1cs.RO

TL;DR

Industrial steel-scrap reconstruction requires accurate registration, but feature correspondences can fail on smooth, repetitive, occluded, and partially overlapping surfaces. The paper introduces correspondence-free bundle adjustment using an adaptive layered global depth map and softmax layer assignment. Experiments report competitive reconstruction accuracy with robustness and low computational cost across challenging industrial scenarios.

  • Problem

    Feature extraction and explicit correspondence establishment are unreliable in challenging industrial scenes, threatening accurate dense reconstruction for measurement and planning.

  • Method

    The method jointly refines sensor poses and an adaptive layered depth map from projected raw depth constraints, using softmax assignment without explicit feature correspondences.

  • Results

    Experiments on self-collected industrial datasets show consistently competitive reconstruction accuracy, stable robustness, and low computational cost.

  • Takeaways & Limitations

    Adaptive depth layers preserve local structures where multiple surfaces overlap within one projected grid region, supporting reliable metric reconstruction for industrial measurement tasks.

Abstract

from arXiv · show

Robotic processing of irregular steel scrap requires dense 3-D measurement to replace manual visual assessment in hazardous cutting workcells. The reconstructed map is used to estimate piece dimensions, boundary geometry, feasible preheating and cutting regions, and collision-aware torch paths. The reconstruction errors therefore propagate directly to downstream measurement and planning. Existing multi-view registration methods commonly rely on feature extraction and data association to establish correspondences between views. In workcells with smooth metallic surfaces, repeated structures, occlusions, and partial overlaps, however, wrong correspondences may be established, leading to inaccurate pose estimation and distorted reconstruction. This paper presents an adaptive layered depth-map-guided bundle adjustment framework for correspondence-free multi-view point cloud registration. The scene is represented by a global 2.5-D grid, where each cell can adaptively maintain multiple depth hypotheses. Raw depth observations are directly projected into the global map to form depth constraints without explicit feature correspondences. At grid cells where multiple surfaces produce conflicting depths, a softmax-based layer assignment links each observation to compatible depth hypotheses. The resulting nonlinear least-squares formulation jointly refines sensor poses and the layered depth map, with correspondences implicitly induced by the depth-map representation and projection model. Experiments on self-collected industrial datasets show that the proposed method achieves consistently competitive reconstruction accuracy while maintaining robustness and low computational cost in challenging industrial scenarios. We release the open-source code implementation at: https://github.com/YiranZhou-Robotics/ADM-BA.git

I. INTRODUCTION

Industrial multi-view registration must produce a unified, accurate metric map from partial scans in hazardous steel-cutting workcells. The paper addresses correspondence failures by jointly refining sensor poses and an adaptive layered depth map without explicit feature matching.

  • Motivation: Industrial workcells require aligned partial point clouds for dense reconstruction supporting measurement and robotic cutting decisions.Downstream uses include steel-piece dimensions, feasible preheating regions, and collision-aware torch paths.
  • Problem: Smooth metallic surfaces, repeated parts, occlusions, and partial overlap make local features weak or ambiguous, so incorrect matches can cause pose drift and reconstruction errors.These conditions directly challenge feature-based registration and data association.
  • Related work: Correspondence-free occupancy-grid approaches avoid explicit matching but can impose substantial computational cost when dense voxel-map resolution increases.A compact representation is desirable for mainly top-down sensing while preserving local multi-depth structure.
  • Contribution: The proposed framework projects raw depth observations onto adaptive layers over a global 2.5-D grid and jointly refines poses and map values in nonlinear least squares.Softmax-based layer assignment links observations to compatible hypotheses without explicit feature correspondences.
  • Related work: Pairwise strategies estimate relative transformations between scan pairs, allowing errors to propagate through multiview sequences as accumulated drift.Robustness techniques improve tolerance to outliers but do not remove the pairwise limitation.

B. Correspondence-dependent Multi-view Registration

Correspondence-dependent methods use learned or handcrafted features, pairwise constraints, or pose graphs, while correspondence-free methods use implicit map associations. The proposed representation combines global depth mapping with multiple hypotheses per grid cell to compactly preserve depth conflicts and occlusions.

  • Correspondence-dependent Multi-view Registration: Feature-based bundle adjustment jointly refines poses and landmarks but depends on reliable extraction and cross-view data association.This dependence remains difficult in cluttered, repetitive, or low-texture industrial environments.
  • Correspondence-dependent Multi-view Registration: Pose-graph optimization derives global pose constraints from pairwise registrations, so its quality depends heavily on those relative transformations.Pairwise registration errors can therefore affect the global multiview solution.
  • Correspondence-free Registration: Correspondence-free approaches jointly optimize poses and maps, but voxel grids, SDFs, meshes, and surfels may incur overhead or lack industrially targeted depth-conflict handling.The proposed method uses fewer variables than a full volumetric representation while retaining multiple depths per location.
  • Adaptive Layered Depth Map: The registration input is a sequence of point clouds whose corresponding sensor poses are estimated in a shared global coordinate frame.Local points are transformed using pose rotations and translations before map projection.
  • Adaptive Layered Depth Map: The adaptive map is a set of 2-D grids on the global x-y plane, with one or more depth values per cell and layer boundaries initialized from prominent KDE depth peaks.Observations sharing planar coordinates but differing in depth can occupy separate layers.

B. Layer-wise Assignment

The method assigns each projected observation to valid depth-map layers using compatibility scores and softmax weights, while accommodating missing entries and multiple depth hypotheses.

  • The active layer set contains layers with valid depth values at the observation’s projected planar location.
  • Invalid depth entries are represented as −∞ and serve only as masks during layer assignment.
  • Compatibility scores compare each projected observation with candidate depth layers before softmax assignment.
  • The parameter β controls assignment sensitivity; larger values make soft selection approach hard selection of the most compatible valid layer.
  • Soft assignment weights let observations with different depths update different coexisting layer hypotheses, while weights sum to one across layers.

C. Depth Constraint

The depth constraint penalizes disagreement between observed projected depth and the corresponding value in the adaptive layered map, with interpolation supporting arbitrary projected locations.

  • When poses are accurate and the environment is static, each map-layer depth should be close to the observation’s projected depth.
  • The resulting depth residual compares each raw observation with the value read from its corresponding projected grid-cell location.
  • The softmax-weighted depth objective incorporates layer assignment into the constraint.
  • Bilinear interpolation estimates depth at arbitrary projected positions from the four neighboring grid cells.
  • The depth constraint jointly couples the point-cloud sequence, sensor poses, and adaptive depth maps during optimization.

IV. PROPOSED METHOD

The proposed method directly integrates raw depth observations, sensor poses, and an adaptive layered map in a correspondence-free nonlinear least-squares optimization.

  • Raw depth measurements are used directly, avoiding explicit scan-to-scan feature correspondences.
  • The optimization jointly estimates sensor poses and the adaptive layered depth map.
  • The formulation is expressed as a nonlinear least-squares problem based on the depth constraint.
  • The objective combines a depth-constraint term and a depth-map smoothing term, balanced by weights wD and wS.

1) Smoothing Term:

The optimization uses smoothing and weighted residual structure to reduce local depth noise, improve convergence, and solve pose and map updates with Gauss–Newton iterations.

  • Smoothing Term: The smoothing term penalizes large depth variations between neighboring grid cells to suppress local noise and improve convergence.
  • Iterative Solution: The nonlinear least-squares objective can be written as a compact weighted least-squares problem.
  • Iterative Solution: The block-diagonal weight matrix applies soft assignment weights to depth residuals and balances depth and smoothing terms through wD and wS.
  • Iterative Solution: A first-order Taylor expansion around the current estimate provides the local linearization for each iteration.
  • Iterative Solution: The Gauss–Newton update solves for an increment, after which the estimate is updated as Xt+1 = Xt + ∆X.

3) Analytical Jacobian:

The analytical Jacobian separates pose, depth-map, and smoothing derivatives within a joint optimisation that iteratively updates sensor poses and the depth map. Bilinear interpolation supports depth evaluation and gradients at arbitrary projected positions.

  • Analytical Jacobian: The Jacobian comprises pose, depth-map, and smoothing components for the nonlinear least-squares formulation.The three parts are JP, JD, and JS.
  • Analytical Jacobian: Algorithm 1 projects observations, rejects vertical surfaces, initializes the depth map, assigns layers with softmax weights, and jointly updates poses and depths.Iterations continue until the maximum iteration count or update threshold is reached, returning optimal poses and map estimates.
  • Analytical Jacobian: The depth-map Jacobian is obtained from the interpolated depth constraint with respect to neighbouring depth variables.Global JP and JD blocks are stacked over all valid observation-layer pairs.
  • Analytical Jacobian: The smoothing Jacobian is sparse and contains only 1, −1, and 0 entries.This structure applies to the smoothing term with respect to the depth map.

A. Setup

The study uses a self-built structured-light platform and nine datasets spanning controlled, semi-realistic, and realistic industrial conditions. Quantitative evaluation uses physical dimensions and landmark distances, alongside comparisons with five registration baselines.

  • Setup: The platform mounts a structured-light camera on a motorised two-degree-of-freedom base for controlled top-down scanning.The system supports dense 3-D reconstruction for steel-piece measurement, cutting-region localisation, and torch path planning.
  • Setup: Nine datasets cover controlled, semi-realistic, and realistic industrial conditions across three scene groups.Group 1 uses known-dimension objects on plastic film; Group 2 uses a steel-slag bed with fixed landmarks; Group 3 uses randomly placed billets and coils.
  • Setup: Physical object dimensions assess local metric accuracy, while landmark distances assess global alignment and drift.Groups 1 and 2 provide physical references for quantitative evaluation; Group 3 supports qualitative analysis and ablation.
  • Setup: The proposed method is evaluated against five baselines spanning pairwise registration, global optimisation, and BA-based multi-view registration.The comparison includes T+ICP, T+PGO, T+BA, BALM2, and 3D Occ.
  • Setup: Scene 1 qualitative comparisons examine local object boundaries and transitions between objects and the supporting surface.Figure 2 maps panels (a–f) to T+ICP, T+PGO, T+BA, BALM2, 3D Occ., and the proposed method.

D. Quantitative Reconstruction Accuracy

The proposed method achieves competitive reconstruction accuracy across controlled and realistic industrial scenes, while qualitative comparisons show clearer boundaries, overlaps, and local surface transitions than several baselines.

  • The proposed method achieves competitive accuracy across all six controlled-scene measurements without unreliable marked results.Several TEASER-based and feature-based methods show large errors or visually unreliable reconstructions in specific scenes.
  • Qualitative comparisons evaluate misalignment, boundary blurring, and structural discontinuities in representative regions across six reconstruction methods.Scene 3 uses numbered white boxes, while Scene 6 uses enlarged insets for narrow boundaries and overlapping regions.
  • The proposed method achieves the best or second-best result on most metrics in realistic steel-slag scenes, particularly Scenes 5 and 6.Independent landmark and dimension measurements support stable local shape reconstruction and global alignment across scene complexities.
  • Feature-based and TEASER-initialised baselines often leave duplicated edges, thickened boundaries, scan offsets, or local discontinuities despite recovering coarse layouts.These artefacts are especially visible in repeated layouts and overlapping or supporting-surface regions.
  • In Scene 3, the proposed method avoids extra marked artefacts, preserves a continuous side-view profile, and produces cleaner boundaries and smoother supporting-surface transitions.The comparison includes T+ICP, T+PGO, T+BA, BALM2, and 3D Occ. under a repeated billet layout.
  • The adaptive layered depth map preserves local structures in boundary and overlap regions rather than only improving scalar reconstruction errors.These local artefacts directly affect the reliability of reconstructed steel-piece geometry for cutting-region selection and collision-aware motion planning.

F. Ablation on Adaptive Depth Layers

The ablation shows that adaptive depth layers preserve local structures when multiple surfaces overlap a projected grid location. The method also combines this representation with competitive accuracy and substantially lower runtime than the compared alternatives.

  • Ablation on Adaptive Depth Layers: Under identical input data, projection, and optimisation settings, the adaptive layered method preserves clearer and more compact boundaries than the single layered baseline.Scenes 7–9 compare photographs, the single layered depth-map baseline, and the proposed adaptive method in boundary and overlap regions.
  • Ablation on Adaptive Depth Layers: A single depth value can average or merge observations from object sides, supporting surfaces, and overlaps projected to the same grid location.The adaptive map maintains multiple depth hypotheses where depth structure is complex while retaining one effective layer where it is simple.
  • Runtime Analysis: The proposed method processes the full Scene 6 sequence in 24.4 s, versus 500 s for T+ICP, 691 s for T+PGO, 245 s for T+BA, 35 s for BALM2, and 1750.5 s for 3D Occ.The comparison uses the same 21-frame sequence and workstation; reported time includes loading, preprocessing, map initialisation where applicable, and optimisation.
  • Runtime Analysis: The compact adaptive 2.5-D grid preserves local depth discontinuities while avoiding the overhead of a dense 3-D occupancy map.3D Occ. has competitive accuracy in several measurements but substantially higher computational cost.
  • Overall Findings: Across controlled, semi-realistic, and real industrial datasets, the method maintains competitive reconstruction accuracy and stable performance across varying scene complexities.The ablation further indicates that adaptive layers are necessary for preserving local structures when multiple surfaces overlap one projected grid region.
Loading 2609.01089v1…