A Probabilistic Framework for Learning Kinematic Models of Articulated Objects

1. Introduction

• The paper studies robots interacting with articulated domestic objects, where understanding part motion is needed to plan actions such as door-opening trajectories and assess success.Source: §1 · p. 1
• The target is a compact model learned from pose observations that specifies part connectivity, degrees of freedom, and the kinematic function, for reasoning about future or unseen configurations.Source: §1 · p. 1
• The approach learns models from sensor data that describe part connectivity, degrees of freedom, and link constraints, and uses them to control manipulator motion.The paper also describes improving learning through past experience and generalizing to closed-chain objects with kinematic loops.Source: §1 · p. 2
• Earlier approaches to doors and drawers were often model-free or assumed substantial model knowledge, while articulation knowledge supports state estimation, motion prediction, and planning.Source: §1 · p. 2
• The framework extends prior kinematic-tree work to general kinematic graphs, enabling objects with loops to be modeled and effective degrees of freedom to be found directly.A unified framework and expanded experiments are presented, building on earlier work with simulation, motion capture, stereo, and direct interaction.Source: §1 · p. 3

2. A Probabilistic Framework for Articulated Objects

• An articulated object consists of multiple parts connected by passively actuated mechanical links that constrain their relative motion.Examples include a door hinge constraining motion to an arc and a drawer shaft constraining motion to a line segment.Source: §2 · p. 3
• For each pair of parts, the system fits candidate link models to relative motion and selects the kinematic structure that best explains the observations by maximizing posterior probability.Source: §2 · p. 4

2.1 Notation

• A kinematic graph has vertices for object parts and undirected edges for links, each associated with a link model and parameter vector.Nontrivial links can also have a latent configuration variable, such as a door's opening angle, whose dimension represents link degrees of freedom.Source: §2.1 · p. 4
• The graphical model represents repeated observations of link configurations, noisy part poses, and relative transformations used to infer the link model and its parameters.The reduced view collapses link configuration into an object configuration node and displays part connectivity as a kinematic graph.Source: §2.1 · pp. 4–5

2.2 Problem Definition

• The graph posterior is difficult to maximize because it spans a high-dimensional, non-convex space of discrete structures and continuous kinematic properties.The goal is to find the graph most likely given the observed poses.Source: §2.2 · p. 5
• For kinematic trees, independent edges allow link models to be estimated separately from the structure, before the structure itself is selected.The method first estimates models for all possible part pairs, then estimates the kinematic structure.Source: §2.2 · p. 6
• Link fitting estimates each model's parameters by maximizing data likelihood, and Bayesian information criterion (BIC) selects among candidate models.The parameter prior is assumed uniform and dropped, as is the model-specific normalizing constant.Source: §2.2 · pp. 6–7
• The full procedure fits link models, selects a model for each link, and then estimates the graph structure; the tree solution can extend to graphs with loops.Structure estimation uses BIC over candidate structures, approximating the integral with the maximum-likelihood link-model estimates.Source: §2.2 · p. 7

2.3 Observation Model

• The model assumes zero-mean Gaussian noise with covariance Σz and allows a small fraction of observations to be outliers.The outliers may arise from poor perception, bad data association, or other sensor failures.Source: §2.3 · p. 7
• Outliers are treated as unrelated to the true transformation and assigned a uniform likelihood, with a latent indicator distinguishing inliers from outliers.The outlier probability is denoted γ.Source: §2.3 · pp. 7–8
• The likelihood mixes Gaussian inlier and uniform outlier distributions, while the prior over γ favors models with fewer outliers.The prior is proportional to exp(−wγ), where w is a weighting constant.Source: §2.3 · p. 8

2.4 Candidate Models

• Doors and drawers motivate common revolute and prismatic models, while other linkages motivate a broader candidate set.Examples include spherical joints, screws, two-bar links, and garage doors.Source: §2.4 · p. 8
• The candidate set includes revolute, prismatic, and rigid models, plus a parameter-free Gaussian-process model for nonstandard articulations.The nonstandard model combines dimensionality reduction with Gaussian process regression.Source: §2.4 · p. 8, Table 1 · p. 9
• Because the connection type is initially unknown, the framework fits all candidate models to the observations and selects the best one.The models are deterministic; observation noise accounts for uncertainty in measured part poses.Source: §2.4 · p. 8

2.5 Model Fitting using Maximum Likelihood Consensus

• MLESAC estimates parameters from minimal random samples, scores the full observation sequence, then refines the best guess with BFGS while estimating the outlier ratio.The procedure addresses the sensitivity of least squares to noise and outliers.Source: §2.5 · p. 9

2.5.1 Rigid Model

• The rigid model represents a link with a fixed relative transformation, has six parameters, and requires no inverse kinematics because it has zero degrees of freedom.Source: §2.5.1 · pp. 9–10

2.5.2 Prismatic Model

• The prismatic model represents one-dimensional translation along a unit axis from a fixed origin, with the configuration encoding distance along that axis.Its parameter vector has nine dimensions; two observations initialize the origin and axis.Source: §2.5.2 · p. 10

2.5.3 Revolute Model

• The revolute model represents one-dimensional motion along a circular arc, parameterized by a rotation center and a rigid transformation to the moving part.Its 12-dimensional parameter vector is estimated from three observations by finding their plane, rotation axis, and circle center.Source: §2.5.3 · p. 10

2.5.4 Gaussian Process Model

• For links that do not follow simple shifting or rotation axes, the Gaussian-process model learns a forward kinematic mapping from a reduced-dimensional configuration manifold to transformations.Dimensionality reduction assigns configurations to observations; Gaussian processes model the forward mapping without specifying a parametric form and support incremental learning.Source: §2.5.4 · p. 11
• The Gaussian-process link model also supports configuration inference for new transformations and extends to configuration spaces with more than one dimension.PCA is used for near-linear manifolds, while LLE can represent strongly nonlinear manifolds.Source: §2.5.4 · pp. 11–12

• The likelihood of an observation is evaluated by integrating over its latent configuration, assuming independent degrees of freedom and a uniform configuration distribution.Source: §2.6 · p. 12
• Under a unimodal observation model, evaluation approximates the integral at the configuration inferred by inverse kinematics, then predicts a transformation with forward kinematics.Source: §2.6 · pp. 12–13
• The resulting likelihood uses the observation model to compare the observed and predicted transformations, effectively projecting noisy observations onto the model.Source: §2.6 · p. 13

• Model selection compares fitted candidates and chooses the model with the lowest BIC.Source: §2.7 · p. 13
• BIC approximates the model posterior using data likelihood and a penalty based on the model's parameter count and the number of training observations.BIC(M) = −2 log p(Dz | M, θ̂) + k log n.Source: §2.7 · p. 13
• The section points readers to Bishop (2007) for further information on BIC.Source: §2.7 · p. 13

• The connectivity extension finds kinematic trees for objects with multiple parts by selecting which fitted link models belong in the structure.Source: §2.8 · pp. 13–14
• The method considers spanning trees of a fully connected graph, restricting the structure to mechanisms without kinematic loops.Explicitly evaluating all candidate trees is not tractable in practice.Source: §2.8 · p. 14
• Assuming independent links makes the structure posterior factor across edges, so maximizing it becomes a minimum spanning tree problem with edge costs approximated by BIC.The tree can be found in O(p^2 log p) time using Prim’s or Kruskal’s algorithm.Source: §2.8 · p. 14

3. Framework Extensions

• The framework's extensions add prior transfer, closed-chain graphs, and direct estimation of an articulated object's degrees of freedom.These extend the from-scratch model-learning approach described earlier in the section.Source: §3 · pp. 14–15

3.1 Learning and Exploiting Priors

• Prior models are merged with new trajectories when they explain the data better, improving parameter estimation; unexplained motion instead triggers a new model.Trajectories are considered sequentially to avoid the combinatorial cost of evaluating all assignments, and each successful new model becomes prior information for future objects.Source: §3.1 · p. 16

3.2 Closed Kinematic Chains

• A randomized polynomial-time search explores neighboring graph structures and is guaranteed not to perform worse than the initial spanning tree.Closed-chain pose constraints require globally consistent predictions, obtained by optimizing part poses against the link constraints.Source: §3.2 · pp. 17–18

3.3 Finding the Number of DOFs

• Loop closure can make an object's degrees of freedom fewer than the sum for its links; the example has four one-DOF links but only one object DOF.The method maps the link configuration space to a lower-dimensional object configuration space, and structure selection favors fewer DOFs when they explain the data equally well.Source: §3.3 · p. 19

4. Perception of Articulated Objects

• Pose observations for model estimation come from marker-based perception, marker-less perception, or a manipulator's forward kinematics and joint encoders.The observation sequence includes the poses of all parts of the object.Source: §4 · p. 20

4.1.1 Motion Capturing Studio

• The PhaseSpace motion-capture system uses active markers and provides virtually noise- and outlier-free observations, with specified positional noise below 0.005 m and orientation noise below 1°.The system uses high-speed cameras mounted on a ceiling rig.Source: §4.1.1 · p. 20

4.1.2 ARToolkit Markers

• ARToolkit uses a single camera and passive coded markers; at 2 m with 0.08 m markers, typical noise is 0.05 m in position and 15° in orientation.Its noise depends strongly on marker distance and camera angle.Source: §4.1.2 · p. 20

4.1.3 Checkerboard Markers

• OpenCV checkerboards yield typical noise of 0.005 m in position and 5° in orientation at 2 m with 0.08 m markers.Different markers can be distinguished using checkerboards with varying row and column counts.Source: §4.1.3 · p. 20

4.2 Marker-Less Pose Estimation

• Marker-less stereo perception is restricted to kitchen-cabinet fronts: RANSAC segments planes, rectangles are iteratively fitted, and detections are linked into pose tracks.The authors identify arbitrary-object registration in point clouds as an open issue.Source: §4.2 · pp. 21–22

4.3 Perception using the Joint Encoders of a Mobile Manipulation Robot

• During physical interaction, the robot treats the end-effector trajectory as observations, estimates the object's model continuously, and uses its Jacobian to update Cartesian equilibrium points.A hook can sense handle position through contact but not handle orientation; mobile-base localization reduces global positional accuracy to about 0.05 m.Source: §4.3 · pp. 22–23

5.1.2 Model and Structure Selection

• Model selection chose a revolute model for the microwave, a GP model for the garage door, and a two-branch kinematic tree as observations revealed both cabinet drawers moved independently.For the cabinet, the inferred structure changed from a single drawer link to links for both drawers as the upper drawer was observed moving.Source: §5.1.2 · pp. 24–25

5.1.1 Model Fitting

• Revolute and prismatic models predicted microwave-door and drawer motion with errors below 0.001 m and 0.0016 m, while the GP model handled the garage door when parametric models failed.The GP model achieved 0.0020 m error for the door and 0.0017 m for the drawer; BIC penalizes GP over-complexity, favoring specialized parametric models when appropriate.Source: §5.1.1 · p. 24, Table 2 · p. 25

5.1.3 Multi-Dimensional Configuration Spaces

• For a table moved across a plane, the learned model progressed from rigid to 1-DOF and then 2-DOF GP models as the observed motion expanded.The 2-DOF GP model explained full planar movement on a two-dimensional surface.Source: §5.1.3 · p. 26, Figure 10 · p. 28

5.1.4 Additional Examples

• Across additional household objects, learned models matched observed motion, including GP models for a garage-door linkage and a desk lamp; prediction errors were around 0.001 m and 1° with motion capture and 0.003 m and 3° with checkerboards.The learned configuration ranges can be visualized and projected to object poses.Source: §5.1.1 · p. 24, §5.1.4 · pp. 27–29

5.2 Evaluation of Marker-Less Model Estimation

• After 30 observations, model selection converged in all cases to the true model; drawer datasets had predictive accuracy of approximately 0.01 m and 7°, while the door dataset had 0.01 m and 3.5°.The detector found the cabinet drawer in more than 75% of images up to 2.3 m from the camera.Source: §5.2 · pp. 30, 32

5.3 Operating Articulated Objects with Mobile Manipulators

• The robot successfully opened five mechanisms in 37 of 40 trials (92.5%); PR2 learned models from end-effector trajectories achieved average predictive accuracy below 0.002 m, measured by observed-trajectory residual error.It opened the 3 rotary mechanisms in 21 of 24 trials and the 2 linear mechanisms in all 16.Source: §5.3.1 · p. 33, §5.3.2 · p. 33

5.4 Improving Model Estimation Based on Experience

• With prior models, prediction error fell by a factor of three or more after observing 30% to 70% of a new trajectory, and results were insensitive to moderate noise variations.The approach also correctly assigned 37 trajectories to five mechanisms; noise from 0.02 m to 0.20 m yielded similar results to the 0.05 m setting.Source: §5.4 · p. 36

5.5 Detecting Kinematic Loops

• The learner recovered three revolute links and three DOFs for an open yardstick chain, and four revolute links with one DOF for the closed loop.For the closed chain, the learner estimated one DOF after only the first few observations.Source: §5.5 · pp. 36–37
• The heuristic search’s BIC was 4.3% worse than optimal, matched it in 57.5% of cases, and took 82.6% less time; spanning-tree BIC was 35.2% worse.For the yardstick object with p = 4 parts, the spanning tree was the heuristic search’s starting point.Source: §5.5 · p. 37, Figure 21 · p. 39
• Tests on an artificial four-part object with one revolute joint and a step ladder with two joints sharing one DOF correctly estimated kinematic parameters and DOF counts.Detecting loop closures and reduced DOFs reveals a smaller configuration space, useful when a mobile manipulator reasons about possible object configurations.Source: §5.5 · p. 37, Figure 22 · p. 40

5.5.1 Evaluation of Model Selection Robustness

• With true observation noise σtrue_z,pos = 0.05, the revolute model was selected for assumed noise 0.02–0.2; at n = 50, GP was selected at ≤0.02 and rigid above 0.2.The revolute model was selected across an assumed-noise interval spanning a whole magnitude.Source: Figure 23 · p. 41, §5.5.1 · p. 41
• Additional synthetic experiments found that estimators resist normally distributed noise, while MLESAC-based estimators also resist uniformly distributed outliers.When sample count is fixed, a higher assumed observation noise favors simpler models, and a lower assumption favors more complex models.Source: §5.5.1 · pp. 37, 41

5.6 Open-Source Availability

• The source code, documentation, examples, and tutorials are available under a BSD license, with a step-by-step guide to repeat the experiments on a consumer-grade laptop and webcam.Source: §5.6 · p. 41

6. Related Work

• The paper combines system identification, information theory, computer vision, robotics, and control to learn and use articulated-object models.The section introduces these fields as sources of techniques used in the work.Source: §6 · pp. 41–42

6.1 Kinematic Model Fitting

• Earlier robot calibration methods assume a specified kinematic model and initial parameters, whereas this paper estimates both from observations and uses MLESAC for robustness to outliers.MLESAC is described as robust against reasonable amounts of training-data outliers.Source: §6.1 · p. 42

6.2 Kinematic Structure Selection

• Prior structure-learning work targets topology or restricted joint models; this approach exploits tree-edge independence, uses greedy search for closed chains, and applies BIC to balance fit and complexity.The paper also handles multiple-DOF and non-parametric link models.Source: §6.2 · pp. 42–43

6.3 Pose Estimation

• Pose-estimation methods trade marker requirements, model assumptions, and sensor limitations; the paper demonstrates marker-based monocular, marker-less stereo, and proprioceptive tracking.Stereo systems can have dropouts in low-texture or poorly illuminated areas.Source: §6.3 · pp. 43–44

6.4 Operating Articulated Objects

• Many object-operation approaches assume the kinematic model and do not infer structure; alternatives learn planar 2D models or rotary-joint parameters, while this work targets broader unknown objects.The cited approaches include door and drawer operation systems, planar-object learning, and rotary-joint estimation.Source: §6.4 · p. 44

7. Conclusion

• The approach infers rigid-part connectivity and link articulation models using both parametrized and parameter-free representations.Its evaluations cover model estimation, model selection, and structure discovery on synthetic and real data.Source: §7 · p. 44
• The approach applies to a wide range of articulated objects and a variety of sensor modalities.Mobile manipulators can operate unknown articulated objects, learn their models, and improve over time.Source: §7 · p. 45
• The current approach learns kinematic models from static pose observations, leaving dynamic parameters and time-optimal motion planning as future extensions.The authors suggest using velocity or acceleration observations to learn dynamic parameters and support time-optimal trajectories and higher-speed execution.Source: §7 · p. 45