Source-linked AI summary
A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representation
Jiteng Mu, Weichao Qiu, Adam Kortylewski, Alan Yuille, Nuno Vasconcelos, Xiaolong Wang
TL;DR
General articulated-object modeling lacks the mature implicit-function methods available for rigid shapes, particularly because articulation increases variation and complicates generalization. The paper introduces A-SDF with disentangled shape and articulation codes plus Test-Time Adaptation, and reports improved reconstruction, interpolation, extrapolation, unseen-angle generation, and real-world depth-image performance.
Problem
General articulated objects are less developed than rigid objects in implicit shape modeling, while their larger degrees of freedom make generalization to unseen shapes difficult without joint or part priors.
Method
A-SDF uses separate shape and articulation codes with distinct networks, and adapts the shape encoder at test time while fixing the articulation network.
Results
Experiments report improved reconstruction, interpolation, and extrapolation, unseen-instance generation at unseen joint angles, and reliable 3D-shape generation from real-world RBO depth images.
Takeaways & Limitations
The disentangled representation allows articulation-code control while preserving inferred part geometry, supporting animation of unseen instances and generalization beyond synthetic data.
Takeaways & Limitations
The method trains separate networks for each category, and direct joint inference can converge to local minima because the shape and articulation spaces are non-convex.
Abstract
from arXiv · showhide
Recent work has made significant progress on using implicit functions, as a continuous representation for 3D rigid object shape reconstruction. However, much less effort has been devoted to modeling general articulated objects. Compared to rigid objects, articulated objects have higher degrees of freedom, which makes it hard to generalize to unseen shapes. To deal with the large shape variance, we introduce Articulated Signed Distance Functions (A-SDF) to represent articulated shapes with a disentangled latent space, where we have separate codes for encoding shape and articulation. We assume no prior knowledge on part geometry, articulation status, joint type, joint axis, and joint location. With this disentangled continuous representation, we demonstrate that we can control the articulation input and animate unseen instances with unseen joint angles. Furthermore, we propose a Test-Time Adaptation inference algorithm to adjust our model during inference. We demonstrate our model generalize well to out-of-distribution and unseen data, e.g., partial point clouds and real-world depth images.
1. Introduction
A-SDF addresses the underdeveloped problem of general articulated-object modeling by disentangling shape and articulation, enabling reconstruction, prediction, and test-time adaptation without geometric or joint priors.
- A-SDF represents articulated objects with separate shape and articulation codes, reconstructing and predicting 3D shapes under different articulations.Its differentiability supports applications requiring back-propagation through the representation.
- The model targets general articulated objects without prior knowledge of part geometry, articulation status, joint type, joint axis, or joint location.
- Explicit articulation modeling addresses the difficulty of encoding articulation variation and generalizing to unseen instances with unseen joint angles.
- During inference, inferred shape and articulation codes can be separated so changing the articulation code generates the same instance at different articulations.Fixing the shape code preserves part geometry during generation.
- Test-Time Adaptation updates the shape encoder while fixing the articulation network, supporting reconstruction and generation from partial point clouds and real-world depth images.The architecture preserves disentanglement during shape-encoder adaptation.
- The model generalizes to real-world RBO depth images and quantitatively outperforms the baselines.
2. Related Work
Related work largely focuses on static implicit representations, articulated humans, pose estimation, disentangled representations, or test-instance adaptation, leaving generic articulated-object synthesis comparatively under-explored.
- Continuous differentiable implicit functions efficiently model varied topologies, but most prior work addresses static objects and scenes.
- Parametric mesh models estimate articulated human shape and pose but require substantial expert construction, limiting generalization across large object categories.
- Daily articulated objects remain under-explored in both data and approaches, especially without priors for part geometry, labels, or joint properties.
- Prior articulated-object research commonly estimates 6D poses, representing a different focus from generic articulated-shape synthesis.
- Disentangled representations assign different variation types to separate latent factors, with prior work demonstrating meaningful latent spaces for image synthesis.
- Test-time learning has been used to adapt trained models to out-of-distribution inputs across recognition, synthesis, and reconstruction applications.
3. Method
A-SDF represents articulated objects with separate shape and articulation codes, enabling reconstruction, inference, adaptation, and generation across articulations.
- 3.1. Formulation: A-SDF uses a shape code for object-part geometry and a separate articulation code for joint angles.Its shape encoder produces a shape embedding, while the articulation network uses that embedding and the articulation code to deform the shape.
- 3.1. Formulation: The same shape code is shared across all articulated poses of an instance, while articulation codes represent pose-specific joint angles.Training maintains one shape code per instance across its M articulated shapes.
- 3.1. Formulation: The articulated signed distance function maps sampled points and latent codes to scalar SDF values, with the zero level-set representing the 3D surface.The model can optionally predict part labels alongside SDF values when part supervision is available.
- 3.2. Training: Training optimizes network parameters and instance shape codes using per-point SDF regression, optional part classification, and a Gaussian prior on shape codes.The part-classification loss helps disambiguate object parts, while loss coefficients balance the objectives.
- 3.3. Inference: Inference jointly estimates shape and articulation codes by back-propagation with fixed articulation-network parameters, then refines shape estimation while holding articulation fixed.This two-step procedure addresses noisy shape-code estimates arising from non-convex shape and articulation spaces.
- 3.3. Inference: Test-Time Adaptation fine-tunes the shape encoder with both inferred codes fixed, preserving disentanglement and generation ability for out-of-distribution observations.After inferring a shape code, varying the articulation code generates unseen articulations while keeping the part geometry fixed.
4. Experiment
Experiments evaluate A-SDF on reconstruction, articulation interpolation and extrapolation, shape synthesis, partial point clouds, and real-world depth images. Across these settings, the method outperforms DeepSDF and supports articulated-shape generation beyond observed training angles.
- 4.2. Reconstruction: The proposed method yields smaller Chamfer-L1 distances than DeepSDF for reconstruction across all evaluated classes.The representation uses fewer parameters while producing sharper boundaries and richer details; Table 1 reports the comparison.
- 4.3. Interpolation and Extrapolation: The method reliably interpolates articulated shapes, outperforming DeepSDF by a large margin while producing accurate rather than unrealistic deformations.Table 2 provides the quantitative comparison, and Figure 4 visualizes the qualitative difference.
- 4.3. Interpolation and Extrapolation: 0.280 Chamfer-L1 for the proposed method versus 3.396 for DeepSDF on laptop extrapolation beyond training angles.The proposed method generates valid shapes outside the observed angle range, whereas DeepSDF fails.
- 4.4. Shape Synthesis and Part Prediction: The disentangled representation generates unseen-instance shapes at unseen joint angles, while Test-Time Adaptation further reduces synthesis error.The method also estimates joint angles simultaneously, with errors reported in brackets in Table 3.
- 4.5. Test on Partial Point Clouds: The method generalizes to partial point clouds and real-world depth images, where Test-Time Adaptation improves reconstruction and generation performance.These models are not trained on partial point clouds or real-world depth images; the RBO evaluation covers both reconstruction and generation.
5. Conclusions
A-SDF uses a structured latent space and test-time adaptation to represent, infer, and generate articulated objects across articulations and data conditions.
- A-SDF combines structured shape and articulation codes with test-time adaptation to model articulated objects.The method infers shape and articulation simultaneously and adapts to real-world depth images.
- Experiments on seven Shape2Motion categories show improved shape reconstruction, interpolation, and extrapolation performance.
- The disentangled representation enables controlling articulation codes to generate unseen instances at unseen joint angles.
- The method reliably generates 3D shapes from real-world depth images in the RBO dataset.