PaperlayerView exact revision on arXiv

Source-linked AI summary

PoseRBPF: A Rao-Blackwellized Particle Filter for 6D Object Pose Tracking

Xinke Deng, Arsalan Mousavian, Yu Xiang, Fei Xia, Timothy Bretl, Dieter Fox

arXiv:1905.09304v1cs.CVcs.RO

TL;DR

6D pose tracking needs temporal and uncertainty-aware estimates, especially under occlusion and object symmetry. PoseRBPF decouples translation and rotation in a Rao-Blackwellized particle filter and uses a learned auto-encoder codebook for discretized rotations. It tracks full pose distributions, handles arbitrary symmetries without manual labeling, and achieves state-of-the-art results on two benchmarks.

  • Problem

    Single-image pose methods often ignore temporal information and provide unimodal estimates, which are insufficient for uncertainties from occlusions and object symmetries.

  • Method

    PoseRBPF combines Rao-Blackwellized particle filtering with an auto-encoder codebook, sampling translation while estimating fine-grained rotation distributions conditioned on it.

  • Results

    PoseRBPF achieves more accurate 6D pose estimation on the YCB-Video and T-Less benchmarks while representing uncertainty from object symmetries and pose.

  • Takeaways & Limitations

    The framework tracks full 6D pose distributions for objects with arbitrary symmetries without manual symmetry labeling.

Abstract

from arXiv · show

Tracking 6D poses of objects from videos provides rich information to a robot in performing different tasks such as manipulation and navigation. In this work, we formulate the 6D object pose tracking problem in the Rao-Blackwellized particle filtering framework, where the 3D rotation and the 3D translation of an object are decoupled. This factorization allows our approach, called PoseRBPF, to efficiently estimate the 3D translation of an object along with the full distribution over the 3D rotation. This is achieved by discretizing the rotation space in a fine-grained manner, and training an auto-encoder network to construct a codebook of feature embeddings for the discretized rotations. As a result, PoseRBPF can track objects with arbitrary symmetries while still maintaining adequate posterior distributions. Our approach achieves state-of-the-art results on two 6D pose estimation benchmarks. A video showing the experiments can be found at https://youtu.be/lE5gjzRKWuA

I. INTRODUCTION

PoseRBPF addresses the need for temporally informed, uncertainty-aware 6D pose tracking by combining Rao-Blackwellized particle filtering with a learned auto-encoder. It tracks translation and full rotation distributions, including for objects with arbitrary symmetries.

  • Motivation: Image-based 6D pose methods have largely prioritized single-image accuracy while often ignoring temporal information and returning only one pose hypothesis.Temporal data and uncertainty are important for robotic tasks such as grasp planning and active sensing.
  • Motivation: Unimodal pose estimates cannot adequately represent uncertainties caused by occlusions and object symmetries.These conditions can produce multiple orientation hypotheses.
  • Approach: PoseRBPF factorizes pose estimation into translation sampling and discretized rotation distributions within a Rao-Blackwellized particle filter.The rotation space is discretized at 5 degree resolution into 191,808 bins per particle.
  • Approach: The framework combines Rao-Blackwellized particle filtering with a learned auto-encoder network as an efficient observation model.The network maps varied input images toward a common synthetic-image domain and learns feature embeddings.
  • Contributions and results: PoseRBPF tracks full 6D pose distributions for arbitrary object symmetries without manual symmetry labeling.Experiments on YCB-Video and T-Less show uncertainty representation for symmetric objects and more accurate pose estimation.

III. 6D OBJECT POSE TRACKING WITH POSERBPF

PoseRBPF makes 6D tracking tractable by sampling translation while estimating rotation distributions conditioned on each translation hypothesis. A learned codebook and motion model support efficient probabilistic updates over time.

  • A. Rao-Blackwellized Particle Filter Formulation: The tracking objective is to estimate the posterior distribution of an object’s 3D rotation and translation at every frame of an image stream.The formulation uses observations accumulated through the current time step.
  • A. Rao-Blackwellized Particle Filter Formulation: PoseRBPF factorizes the posterior so translation encodes object location and scale, while rotation is conditioned on translation and the images.Translation determines the image center and scale used to estimate rotation from the object’s appearance inside its bounding box.
  • A. Rao-Blackwellized Particle Filter Formulation: For each translation hypothesis, the tracker crops and resizes a region of interest, encodes it, and matches the code against rotation hypotheses in a codebook.The resulting code distances determine the conditional rotation likelihood P(R|Z, T).
  • A. Rao-Blackwellized Particle Filter Formulation: Each particle samples a translation and maintains a discrete orientation distribution with an importance weight.The orientation uses azimuth, elevation, and in-plane rotation bins at 5 degree resolution, totaling 191,808 bins per particle.
  • A. Rao-Blackwellized Particle Filter Formulation: Particles are propagated through a motion model, while rotation updates use an efficient convolution over the discretized rotations.This produces a new particle set from which the 6D pose distribution is estimated.

B. Observation Likelihoods

PoseRBPF decomposes the observation likelihood into rotation and translation terms, using an auto-encoder and synthetic canonical renderings to obtain probabilistic pose evidence.

  • The likelihood P(Zk|Tk, Rk) is estimated through P(Rk|Tk, Zk) and P(Zk|Tk).
  • Single-pose estimators provide no uncertainty and are computationally expensive when evaluating many particle samples.
  • The auto-encoder maps varied real images to synthetic object renderings with constant lighting and no background or occlusion.
  • Training fixes translation to T0 = (0, 0, z)T and uniformly samples rotation to build embeddings for discretized rotations.
  • Reconstruction quality degrades when RoIs shift or change scale, making the auto-encoder suitable for observation likelihood computation.

2) Codebook Matching:

For each translation hypothesis, PoseRBPF crops and encodes an RoI, matches it against rotation embeddings, and derives translation evidence from reconstruction and rotation likelihoods.

  • A translation hypothesis is projected into the image to determine the RoI center using camera focal lengths and principal point.
  • The RoI size scales with depth relative to the canonical training distance, while its square shape makes it rotation-independent.
  • Cosine similarity between the RoI embedding and each codebook entry produces a probabilistic distribution over discretized rotations.
  • Because incorrect translations shift or rescale the object within the RoI, reconstruction quality supplies the translation likelihood.

C. Motion Priors

PoseRBPF propagates translation and rotation distributions with motion priors, updates particles using RoI-based observations, and resamples according to particle weights.

  • C. Motion Priors: A constant-velocity model propagates the 3D translation distribution between frames.
  • C. Motion Priors: The rotation prior is a Gaussian around the previous rotation and is implemented by convolving the prior distribution with a 3D Gaussian kernel.
  • D. 6D Object Pose Tracking Framework: Initialization backprojects a detector bounding-box center, samples depths, and selects the translation with highest initial likelihood.
  • D. 6D Object Pose Tracking Framework: Each frame updates rotation distributions from encoded RoIs and motion priors, then uses translation posteriors as particle weights for systematic resampling.
  • D. 6D Object Pose Tracking Framework: The framework estimates orientation distributions conditioned on translation particles while evaluating translations through their corresponding RoIs.
  • D. 6D Object Pose Tracking Framework: Translation expectation is obtained by averaging particle translations, whereas rotation expectation requires summarizing multimodal distributions before quaternion averaging.
  • D. 6D Object Pose Tracking Framework: Tracking failures are detected when the maximum codebook similarity across particles falls below a predefined threshold.

E. RGB-D Extension of PoseRBPF

The RGB-D extension adds a depth-based observation likelihood by comparing rendered and measured depth within visible regions, while retaining the RGB auto-encoder likelihood.

  • E. RGB-D Extension of PoseRBPF: PoseRBPF extends observation-likelihood computation to depth measurements in addition to RGB images.
  • E. RGB-D Extension of PoseRBPF: The RGB auto-encoder remains responsible for P(Rk|Tk, ZCk), while depth supplies an additional likelihood term.
  • E. RGB-D Extension of PoseRBPF: For each particle, the method renders the object at its hypothesized pose and compares the rendered depth image with measured depth.
  • E. RGB-D Extension of PoseRBPF: A visibility mask identifies rendered pixels whose depth is less than the measured depth plus a noise margin m.
  • E. RGB-D Extension of PoseRBPF: The depth score combines visible depth discrepancy with the object's visibility ratio, and a Gaussian density converts the score into likelihood evidence.

A. Datasets

PoseRBPF is evaluated on YCB Video and T-LESS, which provide complementary RGB-D video settings with annotated 6D poses and dataset-specific evaluation metrics.

  • YCB Video contains RGB-D sequences of 21 textured and textureless household objects in different arrangements.
  • YCB Video uses ADD and ADD-S to quantitatively evaluate predicted 6D object poses.ADD measures average distances between corresponding model points, while ADD-S uses closest-point distances.
  • T-LESS contains RGB-D sequences of 30 non-textured industrial objects and evaluates 20 test scenes.
  • T-LESS is challenging because its objects exhibit varied symmetries and frequent occlusions.
  • T-LESS reports recall of correct 6D poses using VSD with errvsd < 0.3, 20mm tolerance, and visibility above 10%.

B. Implementation Details

The implementation combines object-specific auto-encoders and offline codebooks with efficient GPU likelihood computation; the rotation-coverage comparison examines scissors and foam brick.

  • The auto-encoder is trained separately for each object for 150,000 iterations using Adam with learning rate 0.0002 and batch size 64.
  • Training uses rendered objects at random rotations over random MS-COCO crops, with 128 × 128 image resolution.
  • The reconstruction loss uses the N pixels with largest errors, with N = 2000 for textured and N = 1000 for non-textured objects.
  • Figure 6 compares rotation-coverage percentiles for PoseRBPF and PoseCNN on scissors, an asymmetric object, and foam brick, which has 180° planar rotation.
  • Each object’s codebook is pre-computed offline, while observation-likelihood computation is performed efficiently on a GPU.

C. Results on YCB Video Dataset

On YCB Video and related evaluations, PoseRBPF improves pose estimation through particle-based tracking, depth information, and uncertainty representation, while performance depends on model fidelity and particle count.

  • YCB Video results: PoseRBPF significantly improves YCB Video 6D pose-estimation accuracy when using 200 particles, especially for symmetric bowl and foam-brick objects.
  • YCB Video results: Increasing the number of particles significantly improves accuracy by covering object-scale and translation variations more effectively.
  • YCB Video results: PoseRBPF++ further improves accuracy by sampling half its particles around PoseCNN predictions and half from the previous time step.
  • YCB Video results: Using depth information with the same particle count significantly improves estimated-pose accuracy, while the encoder still receives only RGB images.
  • T-LESS results: On T-LESS, tracking doubles RGB recall over single-image prediction, depth improves recall by around 76%, and PoseRBPF outperforms ICP refinement by 28%.
  • Uncertainty visualization: PoseRBPF represents uncertainty from rotational, mirror, and discrete rotational symmetries in its estimated rotation distributions.

E. Analysis of Rotation Distribution

PoseRBPF represents rotation uncertainty as full distributions, capturing both symmetry-induced multiple modes and uncertainty around each mode. Compared with Gaussian uncertainty from PoseCNN, it provides better coverage for objects with strong symmetries.

  • PoseRBPF tracks full distributions over object rotations rather than a single rotational estimate.This representation preserves multiple plausible orientations simultaneously.
  • Each viewpoint cluster corresponds to a similarity mode, while its variance represents the pose uncertainty within that mode.The distribution therefore separates ambiguity caused by similar appearances from uncertainty around each candidate pose.
  • For the rotationally symmetric bowl, rings represent azimuthal uncertainty, while different rings show elevation uncertainty.
  • For the foam brick with 180° symmetry, PoseRBPF achieves high coverage, whereas PoseCNN fails to produce good rotation estimates farther from its generated estimate.
  • Experiments show that the tracked distributions capture uncertainties from both object symmetry and object pose.The evaluation included household objects and symmetric textureless industrial objects across two benchmark datasets.
Loading 1905.09304v1…