PaperlayerView exact revision on arXiv

Source-linked AI summary

Generalizable 6D Pose Estimation of Textureless Objects with Planar-based Gaussian Splatting

Jie Lu, Hengtan Zhang, Li Gong, Pengpeng Wang, Xianjia Yu, Jinxiang Deng, Tomi Westerlund, Zhongxue Gan, Lirong Zheng, Zhuo Zou

arXiv:2609.07231v1cs.RO

TL;DR

Textureless-object pose estimation without CAD models remains challenging, and existing Gaussian-based methods can lack sufficient geometric constraints. PG-Pose combines planar-regularized Gaussian reconstruction with geometry-driven pose estimation, achieving 94.2% ADD(S)@0.1d on OnePose-LowTexture and demonstrating robotic grasping on an unseen object.

  • Problem

    Generalizable pose estimation for textureless objects remains limited by insufficient geometric constraints, while prior methods can struggle with symmetric or textureless regions.

    Source: §I · p. 1

  • Method

    PG-Pose combines planar-regularized 3D Gaussian reconstruction with geometry-driven 2D–3D pose initialization and Gaussian-rendering refinement.

    Source: §I · pp. 1–2

  • Results

    94.2% ADD(S)@0.1d on OnePose-LowTexture, with a 2.1% improvement over the SOTA GS-based method.

    Source: §V · p. 6

  • Takeaways & Limitations

    PG-Pose demonstrated practical robotic grasping of an unseen object in a dual-arm industrial robot deployment.

    Source: §V · p. 6

    The representation may be less optimal for smoothly curved objects, and real-world robotic validation covers a single representative object.

    Source: §V · p. 7

Abstract

from arXiv · show

Estimating the 6D pose of textureless objects without prior CAD models remains a critical challenge due to the lack of appearance features. While recent generalizable approaches alleviate the dependence on object-specific models, their performance on low-texture objects is often limited by insufficient geometric constraints in the underlying representations. In this work, we propose PG-Pose, a geometry-aware framework combining Planar-based Gaussian Splatting (PGS) reconstruction and Geometry-driven pose optimization. In the offline representation extraction stage, three distinct representations of the object are extracted from multi-view reference RGB images with known poses. PG-Pose reconstructs a 3D Gaussian representation and renders high-fidelity depth maps to generate 3D point clouds through back projection. In the online pose inference stage, the initial pose of the input image is estimated by 2D-3D correspondence matching between the input image and the reconstructed 3D point clouds, followed by a PGS-Refiner for iterative pose optimization. Evaluations on the OnePose-LowTexture datasets, PG-Pose achieves an average accuracy of 94.2% ADD(S)@0.1d, with a 2.1% improvement average accuracy compared with the state-of-the-art (SOTA) GS-based approach. To further demonstrate the effectiveness of PG-Pose for industrial robots in grasping tasks, we deploy it on a dual-arm industrial robot and successfully realize the grasping task on an unseen object.

I. INTRODUCTION

Textureless-object pose estimation remains difficult because existing approaches generalize imperfectly and Gaussian-based methods can lack reliable geometric constraints. PG-Pose combines planar-regularized Gaussian reconstruction with geometry-driven pose initialization and refinement, achieving strong benchmark accuracy and demonstrating robotic grasping on unseen objects.

I. INTRODUCTION

  • Object-specific methods depend on instance data, while category-level approaches can lose accuracy with substantial shape and scale variation, especially for symmetric or geometrically ambiguous objects.The introduction identifies these limitations as challenges for pose estimation on unseen objects.Source: §I · p. 1
  • Its online stage matches 2D–3D correspondences for PnP-based pose initialization, then refines the initial pose using differentiable Gaussian rendering.The geometry-driven initialization provides a more stable starting point than retrieval-based initialization for symmetric and textureless objects.Source: §I · pp. 1–2
  • PG-Pose injects planar priors, surface-normal consistency, and coplanarity regularization into Gaussian Splatting optimization without relying on CAD models.The geometrically enhanced representation renders depth maps that are back-projected into dense point clouds with improved structural completeness, particularly under low-texture conditions.Source: §I · pp. 1–2
  • 94.2% ADD(S)@0.1d on OnePose-LowTexture, with geometry-driven initialization and Gaussian-based refinement combined in a coarse-to-fine framework.The reported robotic grasping demonstration used unseen objects.Source: §I · p. 2

II. RELATED WORK

Textureless pose estimation has moved from depth-based geometry toward multi-view RGB and learned representations, but ambiguity and appearance limitations persist. The reviewed methods trade reliance on depth, keypoints, or sparse geometry, motivating stronger geometric constraints.

A. Textureless Pose Estimation

  • Depth-based methods use geometric measurements, but sensor noise and occlusions challenge point-pair matching, while dependence on high-quality depth limits RGB-only use.The review also notes that depth-based networks incorporate geometric priors into neural architectures.Source: A. Textureless Pose Estimation · p. 2
  • Multi-view RGB approaches reduce single-view depth ambiguity, yet keypoint detection can be unreliable on textureless objects and sparse point clouds can leave symmetric or planar regions ambiguous.OnePose++ uses detector-free feature matching to reconstruct sparse point clouds.Source: A. Textureless Pose Estimation · p. 2
  • Neural implicit and Gaussian-splatting methods offer high-fidelity rendering, but the review says stronger geometric constraints are still needed to resolve textureless-object pose ambiguities.This follows the review's account of sparse representations failing to resolve ambiguity in symmetric or planar regions.Source: A. Textureless Pose Estimation · p. 2

B. Gaussian Splatting for Generalizable Pose Estimation

Generalizable pose estimation has used CAD models, point clouds, and Gaussian Splatting representations. PG-Pose builds on Gaussian-based methods by regularizing geometry, then uses offline representations in a cascaded online inference process.

B. Gaussian Splatting for Generalizable Pose Estimation

  • Traditional generalizable pose representations include predefined CAD models and point clouds, while 3D Gaussian Splatting provides explicit, high-fidelity rendering.The passage describes 3DGS as a major development in scene representation.Source: B. Gaussian Splatting for… · p. 2
  • PG-Pose differs from prior Gaussian-based approaches by enforcing coplanarity and normal alignment during reconstruction, rather than relying primarily on photometric loss.The cited account notes that GS-Pose uses retrieval-based initialization and that GS2Pose and iG-6DoF address refinement or initialization through other designs.Source: B. Gaussian Splatting for… · pp. 2–3

III. PROPOSED FRAMEWORK

  • The framework constructs object representations offline once per object, then estimates and refines a query image's pose online using those prebuilt representations in a cascade.The framework overview identifies offline representation extraction and online pose inference as its two stages.Source: §III · p. 3

A. Offline Representation Extraction

Offline extraction builds object-centered semantic features, a planar-constrained Gaussian model, and a point cloud from rendered depth. Plane detection and iterative refinement supply geometric structure for the representation.

A. Offline Representation Extraction

  • Object semantic tokens are extracted from multi-view reference images, with keyframes selected by farthest-point sampling before co-segmentation and object-feature masking.The co-segmenter uses center tokens and stacked self- and cross-attention; its decoded masks preserve object-centric features for reconstruction.Source: A. Offline Representation Extraction · p. 3
  • The planar-based Gaussian representation parameterizes each Gaussian with position, rotation, anisotropic scale, opacity, and spherical-harmonics coefficients.Rotations are aligned with detected planar normals, and planar geometric constraints are integrated into the Gaussian Splatting representation.Source: A. Offline Representation Extraction · p. 4
  • Plane normals are estimated from back-projected point-cloud regions, and Gaussian orientations are initialized and refined through alternating plane detection and Gaussian optimization.Coplanarity regularization encourages Gaussians within a plane to share consistent orientation; the detected planar regions are mainly local and do not explicitly enforce graph connectivity.Source: A. Offline Representation Extraction · p. 4
  • Back-projected depth maps provide view-consistent, visibility-aware points for plane detection, filtering low-opacity or negligible Gaussian contributions.The point-cloud representation is reconstructed by perspective-projecting pixels from depth maps rendered from the planar-based Gaussian model; the camera intrinsic matrix is used in the projection.Source: A. Offline Representation Extraction · p. 4

B. Online Pose Inference

PG-Pose initializes a query object's pose from 2D–3D correspondences and refines it through differentiable rendering-based optimization.

B. Online Pose Inference

  • Query-image keypoints are matched to descriptors on a rendered template point cloud to establish 2D–3D correspondences.SuperPoint extracts query and rendered-image descriptors; rendered depth maps are back-projected to form the template cloud.Source: B. Online Pose Inference · p. 4
  • PnP with RANSAC uses the correspondences to estimate a coarse, geometrically consistent initial pose.RANSAC provides robustness against outliers.Source: B. Online Pose Inference · p. 4
  • The GS-Refiner optimizes photometric and geometric consistency while keeping the Gaussian representation fixed and updating only the 6-DoF pose.Differentiable rendering-based refinement corrects residual errors from PnP initialization.Source: B. Online Pose Inference · p. 4

A. Datasets and Evaluation Metrics

The evaluation uses OnePose-LowTexture, a benchmark for low-texture objects, and ADD(S)-based pose accuracy measures.

A. Datasets and Evaluation Metrics

  • OnePose-LowTexture is a standardized benchmark designed for 6D pose estimation of low-texture objects.It contains 40 everyday objects with low or no texture, covering industrial sorting and domestic service robotics scenarios.Source: A. Datasets and Evaluation Metrics · p. 4
  • ADD(S) measures average 3D-point distance after transformation by the ground-truth and predicted poses, with lower values indicating better alignment.Its symmetry-aware minimum compares each predicted transformed vertex with geometrically equivalent model vertices.Source: A. Datasets and Evaluation Metrics · p. 5
  • ADD(S)@0.1d reports the recall rate for ADD(S) within 10% of the object's diameter.The definition of ADD(S) uses the complete set of model vertices and the ground-truth and predicted rigid transformations.Source: A. Datasets and Evaluation Metrics · p. 5

B. Training Details

PG-Pose trains its planar-based Gaussian representation with a joint reconstruction objective and performs pose-only online refinement.

B. Training Details

  • Offline reconstruction jointly optimizes photometric, normal-alignment, and coplanarity losses.The photometric loss enforces pixel-wise appearance consistency; coplanarity regularization penalizes deviations from detected planar regions.Source: B. Training Details · p. 5
  • Coplanarity gradients update Gaussian rotation through its differentiable normal direction, aligning normals while preserving in-plane rotation freedom.Plane normals are estimated from the back-projected point cloud.Source: B. Training Details · p. 5
  • The reconstruction objectives are balanced by scalar weights, and Table I reports 6D pose-estimation results on OnePose-LowTexture datasets; “init” denotes initial pose-estimation results.All networks and reconstruction components are optimized with Adam, using an initial learning rate of 1 × 10−3 and weight decay of 1 × 10−4.Source: B. Training Details · p. 5, Table I · p. 5
  • At inference, online refinement minimizes photometric, silhouette, and geometric losses while keeping the Gaussian representation fixed.Only the 6-DoF pose parameters are updated.Source: B. Training Details · p. 5

C. Object Pose Estimation Performance Evaluation

PG-Pose achieves the strongest reported average accuracy on OnePose-LowTexture, with particularly large gains on several textureless or symmetric objects. Its results remain comparable to or modestly better than the baseline on relatively textured objects.

C. Object Pose Estimation Performance Evaluation

  • 94.2% average ADD(S)@0.1d; PG-Pose achieves the best performance across most OnePose-LowTexture categories.OnePose++ reaches 88.9%, while GS-Pose shows a large gap between initial and refined pose accuracy.Source: C. Object Pose Estimation Performance… · p. 5
  • 5.1% on Cam and 4.8% on Shin, with a 3.6% gain on Cat over the baseline.These objects are identified as textureless and symmetric.Source: C. Object Pose Estimation Performance… · p. 5
  • PG-Pose delivers modest improvements on relatively textured Toy and Tea, and performs strongly on partially textured David and self-occluded Marse.The reported results suggest more reliable convergence under difficult viewing conditions.Source: C. Object Pose Estimation Performance… · p. 6

D. Real World Experiment on Grasp Tasks

In real-world trials, PG-Pose outperforms OnePose++ on grasp success for a single unseen textureless bottle. Ablations on OnePose-LowTexture report declines when planar priors, coplanarity regularization, PnP initialization, or iterative refinement are removed.

D. Real World Experiment on Grasp Tasks

  • 92.3% grasp success for PG-Pose versus 78.5% for OnePose++ across 50 trials on a single unseen bottle.The setup uses a parallel-jaw gripper and restricts evaluation to one reliably graspable textureless object.Source: D. Real World Experiment on Grasp Tasks · p. 6

E. Ablation Study

  • 3.4% lower average accuracy without planar priors, when relying only on photometric consistency.The ablation identifies planar priors as important for textureless surfaces.Source: E. Ablation Study · p. 6
  • Removing coplanarity regularization decreases performance, especially on symmetric objects.Source: E. Ablation Study · p. 6
  • Replacing PnP initialization with retrieval reduces initial-pose accuracy and propagates errors into refinement.Source: E. Ablation Study · p. 6
  • 82.7% average accuracy without iterative Gaussian-based refinement, indicating its role in final pose alignment.Source: E. Ablation Study · p. 6

V. CONCLUSIONS

PG-Pose uses planar priors to enhance 3D Gaussian geometry for CAD-free pose estimation of textureless objects. It achieves 94.2% ADD(S)@0.1d on OnePose-LowTexture, while the real-world validation and the representation's suitability have stated scope limits.

V. CONCLUSIONS

  • 94.2% ADD(S)@0.1d on OnePose-LowTexture, a 2.1% improvement over the SOTA GS-based method.The conclusion attributes the approach's geometric cues for initialization and refinement to planar priors that alleviate low-texture photometric ambiguity.Source: §V · p. 6
  • The locally planar representation may be less optimal for objects dominated by smooth curved surfaces, such as spheres or highly cylindrical shapes.Source: §V · p. 7
  • Real-world robotic validation is limited to one representative object; broader object categories and manipulation tasks remain future work.The stated constraint is the employed end-effector.Source: §V · p. 7

Related papers

Loading 2609.07231v1…