PaperlayerView exact revision on arXiv

Source-linked AI summary

MoRE: Motion-Regularized Encoding for Articulated 3D Generation

Guantian Zheng, Shan Jin

arXiv:2610.30337v1cs.CV

TL;DR

A plausible-looking single-image reconstruction may still encode the wrong hinge or translation, and reconstructing joint attributes alone does not test induced motion. MoRE couples joint-attribute learning to anchor-motion supervision and generates articulated states from one rest-pose Gaussian object. It reports stronger articulated-state geometry and visual similarity than compared methods, while retaining a rest-state reconstruction limitation.

  • Problem

    A single-image articulated reconstruction must recover appearance and part motion, since plausible geometry or joint attributes alone do not establish that the object moves correctly.

    Source: §I · p. 1

  • Method

    MoRE combines an axis-orthogonal joint-line encoding with differentiable expected anchor-motion supervision, image-conditioned functional latents, and part-bound Gaussian appearance.

    Source: §I · p. 1, A. Functional voxel representation · p. 2

  • Results

    MoRE ranks first on four of six metrics on PartNet-Mobility and five on ACD; compared with PAct, AS-CD decreases from 0.0176 to 0.0092 and from 0.0218 to 0.0145, respectively.

    Source: A. Comparison with prior methods · p. 5

  • Takeaways & Limitations

    Controlled ablations show motion supervision reduces anchor-motion error with either encoding, and the combined design gives the lowest errors across all six metrics.

    Source: §V · p. 6

    The motion model assumes coordinate-aligned axes, evaluates part motions independently, and does not resolve inference-time clustering or cross-grid assignment errors.

    Source: E. Limitations · p. 6

Abstract

from arXiv · show

Generating an articulated object from a single image requires recovering both its appearance and the mechanism governing its parts. A compact encoding of joint attributes does not by itself ensure that reconstruction preserves the resulting motion. We present MoRE, Motion-Regularized Encoding for articulated 3D generation. MoRE represents coordinate-aligned joint lines with axis classification and two axis-orthogonal Pl\"ucker moment coefficients, removing the arbitrary choice of an origin along the axis. During representation learning, decoded joint hypotheses are used to move corresponding three-dimensional anchors, and their motions are compared with those of annotated joints. A differentiable expectation over joint types and axis directions couples attribute reconstruction to the motion it induces. An image-conditioned flow learns the resulting functional latents, while a multi-state-adapted Gaussian model supplies appearance. At inference, recovered part assignments and joints articulate a single set of rest-pose Gaussians without regenerating the object for each state. Experiments on PartNet-Mobility and ACD show stronger part-box agreement, articulated-state geometry and visual similarity than the compared baselines, while joint prediction reduces axis, origin and range errors relative to ArtiLatent. Controlled ablations support the joint-line encoding and show that motion supervision reduces anchor-motion error. Code and pretrained checkpoints will be released in the future.

I. INTRODUCTION

MoRE addresses the gap between reconstructing joint attributes and preserving the motion they induce. It combines an axis-orthogonal joint-line encoding with motion supervision and an image-conditioned generation pipeline.

I. INTRODUCTION

  • MoRE trains sparse functional latents with both attribute reconstruction and analytic motion supervision.The motion regularizer evaluates how decoded joint hypotheses move fixed anchors against annotated joints.Source: §I · p. 1
  • An image-conditioned flow learns functional latents, while a multi-state-adapted Gaussian appearance model supports articulation of the same decoded object across states.At inference, predicted part assignments connect recovered joints to the appearance model.Source: §I · p. 1
  • Axis classification and two orthogonal Plücker moment coefficients specify coordinate-aligned joint lines without an arbitrary along-axis origin.A point shifted along the joint axis represents the same rotation, so attribute-wise reconstruction alone does not measure the combined effect of errors on moving geometry.Source: §I · p. 1

II. RELATED WORK

Prior work spans static image-conditioned 3D representations, articulated generation, and reconstruction from observations across states. MoRE connects functional joint attributes to their induced motion and part-bound Gaussian appearance.

II. RELATED WORK

  • Static sparse-latent methods provide geometry and appearance priors, but do not specify the parts and joints required for articulation.MoRE associates functional attributes with sparse support and connects the resulting parts to Gaussian appearance.Source: §II · pp. 1–2
  • Articulated-generation methods use part geometry, graphs, structured latents, or pretrained 3D priors; MoRE supervises joint-line attributes together with the analytic motion they induce.Plücker joint representations also appear in NAP; MoRE uses a classified coordinate axis and two coefficients in its orthogonal plane to specify the line.Source: §II · p. 2

III. METHOD

  • Unlike reconstruction methods that exploit observed state changes, MoRE infers unobserved structure from a single image and adapts appearance using multi-state observations during training.Its functional voxel representation stores descriptors, joint type, axis class, moment, and limits; the joint line uses an axis-orthogonal moment encoding.Source: §II · p. 2, A. Functional voxel representation · p. 2

B. Functional latent generation

MoRE learns functional latents by reconstructing voxel attributes and regularizing the motion of decoded joint hypotheses. An image-conditioned flow learns those latents, and inference recovers part-level joints from predicted voxel attributes.

B. Functional latent generation

  • The functional VAE combines weighted attribute reconstruction with a KL-divergence term, using classification for semantic labels, joint types, and axes and regression for boxes, moments, and limits.Axis supervision applies only to movable voxels because fixed parts have no motion axis.Source: B. Functional latent generation · p. 2
  • During motion-regularized VAE training, decoded part-level joint hypotheses move fixed voxel-center anchors at normalized progress values, with predicted and annotated limits interpolated separately.The expected squared motion discrepancy spans soft joint-type and axis hypotheses; errors are squared before averaging, and part and object means balance voxel counts.Source: B. Functional latent generation · pp. 2–3
  • After VAE training, a transformer flow receives sparse latent tokens, time, and image features and learns the velocity field from noisy latents toward clean latents.Sampling integrates the learned velocity from noise at t = 1 toward t = 0, where the decoder recovers the functional field on generated support.Source: B. Functional latent generation · p. 3
  • At inference, voxel labels are grouped by semantic class and clustered by bounding-box descriptors; part-level votes and averages recover joint attributes and a shared rigid motion.The resulting part count comes from clustering, while ground-truth parts supervise training only.Source: B. Functional latent generation · p. 3

C. Articulation-aware appearance

MoRE adapts a pretrained Gaussian appearance model using observations across articulated states, so motion-revealed interior surfaces are represented. The adapted appearance flow then supplies latents for image-conditioned generation.

C. Articulation-aware appearance

  • Multi-state observations adapt the pretrained TRELLIS appearance encoder and Gaussian decoder because opening parts reveals interior surfaces absent from closed-state images.Offline DINOv2 features are aggregated onto rest-pose voxels, and each predicted Gaussian inherits its voxel's part assignment.Source: C. Articulation-aware appearance · p. 3
  • Training renders decoded Gaussians after known-joint transforms and optimizes an objective that includes a DSSIM term and Gaussian-scale regularization.Supervising multiple states exposes the decoder to exterior and motion-revealed surfaces; the appearance flow is fine-tuned to the updated latent distribution.Source: C. Articulation-aware appearance · p. 3
  • The ablation metrics distinguish line distance from origin error and report anchor-motion RMSE separately for revolute and prismatic parts.Axis and rotational-range errors are in degrees; other distances use normalized object units.Source: C. Articulation-aware appearance · p. 4
  • At inference, a learnable adapter conditions the appearance flow, while the shared DINOv3 encoder remains frozen.DINOv2 is used offline to construct appearance training latents.Source: C. Articulation-aware appearance · p. 4

D. Part-wise motion and inference

MoRE represents joint displacement relative to the rest pose and uses recovered part assignments and joints to articulate the decoded Gaussians. Each prescribed state is rendered by transforming the same appearance representation.

D. Part-wise motion and inference

  • Joint displacement q_k is measured in radians for revolute joints and normalized object-coordinate distance for prismatic joints.A Gaussian center assigned to part k follows the corresponding joint transform.Source: D. Part-wise motion and inference · p. 4
  • At inference, part clustering recovers joints and assignments, which are transferred to appearance voxels and Gaussians for rigid part transforms.The functional field and appearance are not regenerated for each prescribed joint state.Source: D. Part-wise motion and inference · p. 4
  • Figure 3 shows three objects across four configurations, with known joints articulating Gaussians decoded from encoder latents.Each reference image is shown above its reconstruction in a pair.Source: Figure 3 · p. 5
  • VAE encoders and multi-state observations are used during training, while image-conditioned flows supply latents during generation.Source: D. Part-wise motion and inference · p. 5

A. Comparison with prior methods

On PartNet-Mobility and ACD, MoRE ranks first on most reported single-image generation metrics and improves articulated-state geometry over PAct. Its joint prediction also reduces axis, origin, and range errors relative to ArtiLatent.

A. Comparison with prior methods

  • MoRE ranks first on four of six metrics on PartNet-Mobility and five on ACD; compared with PAct, AS-CD falls from 0.0176 to 0.0092 and from 0.0218 to 0.0145, respectively.CLIP reaches 0.9545 and 0.9401 on the two datasets, respectively.Source: A. Comparison with prior methods · p. 5
  • Rest-state surface reconstruction remains a limitation: RS-CD is higher than PAct on both datasets, and PartNet-Mobility AOR is not the best.Source: A. Comparison with prior methods · p. 5
  • MoRE has lower axis, origin, and both range errors than ArtiLatent, with axis error decreasing from 1.14° to 1.03° and origin error from 0.10 to 0.04.Type accuracy is 99.16%, 0.01 percentage points below ArtiLatent's 99.17%.Source: A. Comparison with prior methods · p. 5

B. Experimental setup

The experiments assess single-image generation and joint-line/motion ablations on public benchmarks, alongside separate component evaluations using procedural functional data and multi-state appearance observations.

B. Experimental setup

  • Public single-image benchmarks use RS/AS part-box distance, surface CD, part overlap, and CLIP similarity; axis in the joint and ablation tables is angular error in degrees.Earlier component axis-classification accuracy is a separate percentage metric.Source: B. Experimental setup · p. 5
  • Functional learning uses 499,329 procedural objects across 18 Infinigen-Articulated categories; the shape-conditioned component evaluation uses 144 held-out objects without functional ground truth.The earlier functional evaluation uses DINOv3 conditioning and a supplied part count K.Source: B. Experimental setup · p. 5
  • Appearance adaptation uses 3,276 SINGAPO-derived objects rendered in eight states from 48 views; reconstruction measurements use asset joints and encoder latents.The teaser and qualitative image-only generation use image input and are separate from these component measurements.Source: B. Experimental setup · p. 5
  • With moment encoding fixed, motion supervision reduces RMSE-R/P from 0.0431/0.0207 to 0.0362/0.0168, while the combined design yields the lowest errors across all six metrics.The ablation compares moment encoding with an origin-point control and changes only the motion-loss weight in its primary comparison.Source: B. Experimental setup · p. 5

C. Earlier component measurements

The measurements report attribute-only joint prediction scores and diagnose appearance reconstruction, including its sensitivity to coordinate alignment and color quality. The appearance diagnostics isolate residual error in the appearance pathway rather than providing an inference method.

C. Earlier component measurements

  • 96.17% type accuracy, 92.35% axis accuracy, 94.54% movable F1 and latent MSE 0.460 were obtained by the attribute-only baseline on 144 held-out objects.These image-and-shape-conditioned scores use a supplied part count K and do not measure the new motion regularizer.Source: C. Earlier component measurements · p. 5

D. Appearance reconstruction

  • Correcting a 90° frame mismatch raises mean voxel-silhouette IoU from 0.617 to 0.876 over 20 objects, with improvement on 19.The passage identifies Table IV as the visual for the PSNR and IoU improvement.Source: D. Appearance reconstruction · p. 5, Table IV · p. 6
  • Replacing only Gaussian colors with ground-truth back-projections improves PSNR by 2.79 dB while IoU changes by 0.007, locating residual error in appearance.Color substitution is a diagnostic reference, not an inference method; Figure 3 shows four configurations with appearance attached to the same parts.Source: D. Appearance reconstruction · pp. 5–6

E. Limitations

The model assumes coordinate-aligned axes and evaluates part motions independently, while inference remains vulnerable to assignment errors. The conclusion reports MoRE's comparative results across articulated-state geometry, visual similarity, and joint prediction.

E. Limitations

  • The motion model covers fixed, revolute and prismatic parts, assumes coordinate-aligned axes, and evaluates part motions independently.Training with annotated part groups does not resolve clustering or cross-grid assignment errors at inference; run-to-run variability is unquantified, and density and friction are stored rather than calibrated physical predictions.Source: E. Limitations · p. 6

V. CONCLUSION

  • MoRE obtains the best articulated-state CD and CLIP scores among compared methods on PartNet-Mobility and ACD, with lower axis, origin and range errors than ArtiLatent and comparable type accuracy.The combined design gives the lowest errors across all six metrics, and controlled ablations show motion supervision reduces anchor-motion error with either encoding.Source: §V · p. 6
  • MoRE combines an axis-orthogonal joint-line encoding with a differentiable motion objective, image-conditioned functional latents, and part-bound Gaussian appearance for explicit articulation.The joint-line encoding removes the arbitrary along-axis origin choice.Source: §V · p. 6
Loading 2610.30337v1…