DragAPart: Learning a Part-Level Motion Prior for Articulated Objects

• Part-level dragging changes articulated components, such as opening a cabinet door, rather than repositioning the entire object.Source: §1 · pp. 1–2
• A generalist motion model should represent physical configurations and transitions across humans, animals, deformable materials, and furniture without a universal deformable template.Source: §1 · p. 2
• A drag moves one physical point while the model infers plausible, minimally coupled motion for related parts, such as opening one door without moving independent drawers.Source: §1 · p. 2
• Existing image generators respond to drags mainly by shifting whole objects, motivating methods that produce more fine-grained dragging outcomes.Source: §1 · p. 2
• Drag-a-Move provides synthetic part-level drag data for fine-tuning generators, while texture randomization improves generalization to real images and unseen categories despite limited object count.Source: §1 · pp. 2–3
• DragAPart introduces a new drag encoding and studies both transformer and pretrained latent-diffusion architectures for part-oriented drag control.Source: §1 · p. 3
• The model supports downstream articulated-object motion-parameter optimization and drag-prompted segmentation of moving parts.Source: §1 · p. 3

• Generative-model conditioning is central to controlling generated content and quality, including through controls injected into pretrained diffusion models.Source: §2 · p. 3
• Existing drag-conditioned methods include training-free latent optimization, guidance-based diffusion, task-specific training, and extensions of text-to-video generators.Source: §2 · p. 3
• Many prior methods achieve object repositioning but do not control motion at the level of individual object parts.Source: §2 · p. 3
• DragAPart addresses part-level motion by introducing a synthetic dataset and a new encoding technique for drags.Source: §2 · p. 4

3.1 Preliminaries

• DragAPart models P(x|y, D), generating an object image that satisfies drag constraints while preserving physically plausible, part-level motion.Multiple drags impose simultaneous constraints, while stochastic prediction handles under-specified or inconsistent user inputs.Source: §3.1 · pp. 4–5
• Pre-trained image generators often move the whole object rather than learn nuanced part interactions, motivating a dedicated motion prior for articulated objects.The paper attributes this difficulty partly to training data entangling viewpoint and multi-object interactions.Source: §3.1 · p. 5, §3.2 · p. 5

3.2 DragAPart: Architecture and Drag Encoding

• DragAPart fine-tunes a latent diffusion model on synthetic triplets and injects reference-image and drag conditions into its denoising network.The model supports both Stable Diffusion’s U-Net and DiT architectures; the reference image supplies cross-attention keys and values, while drag features enter each block.Source: §3.2 · pp. 5–6
• The multi-resolution flow encoder separates drag sources, terminations, and drag instances across channels, reducing interference and enabling conditioning at compatible network resolutions.Each drag is encoded in both source-to-termination and swapped directions, with padding when fewer than the maximum number of drags is present.Source: §3.2 · pp. 6–7

3.3 Generalization to Real Data via Domain Randomization

• Randomizing object-part textures during synthetic training significantly improves generalization to real-world images and out-of-domain categories.The strategy discourages the model from relying on fixed synthetic pixel values rather than understanding correspondence to the reference image.Source: §3.3 · p. 7

3.4 Downstream Applications

• DragAPart’s motion prior supports moving-part segmentation and articulated-motion analysis, complementing methods that reposition entire objects.Segmentation compares denoiser features from conditioned and unconditioned forward passes, while motion analysis estimates motion parameters by rendering candidate transformations.Source: §3.4 · p. 8

4 The Drag-a-Move Dataset

• Drag-a-Move represents each example as a triplet in which y is the initial object image, D contains applied drags, and x is the compatible resulting state.The dataset aims for minimal state changes, so each drag affects the smallest subset of parts needed for physical plausibility.Source: §4 · pp. 8–9
• The dataset is built from GAPartNet assets with rich part-level annotations, selecting 763 models across 16 categories containing hinge or slider parts.Eligible part labels include Hinge Handle, Slider Drawer, Hinge Lid, and Hinge Door.Source: §4 · p. 9
• Animations vary which joints move and whether stationary parts remain at rest or start from random articulation states.Moving joints transition between extreme or randomly selected states, producing cases where only subsets of parts change.Source: §4 · p. 9
• Sparse drags are generated by projecting visible points on moving parts from their original to displaced positions, selecting one trajectory per moving joint.Sampled points are weighted by displacement, and drag combinations can be generated on the fly during training.Source: §4 · pp. 9–10
• Rendering produces 48 animations of each asset with 36 frames under random lighting and viewpoints, yielding 40 million unique training image pairs.A second version assigns each object part a random monochromatic texture to improve generalization.Source: Figure 3 · p. 9, §4 · p. 10

5.1 Datasets and Implementation Details

• The evaluation spans unseen objects and categories in Drag-a-Move, real CO3D and ABO images with manually defined drags, and Human3.6M actors held out by identity.The synthetic test split contains 150 image pairs from 23 unseen objects across 10 categories; an O.O.D. split uses Blender renderings and point-tracked drags.Source: §5.1 · p. 10

5.2 Comparisons with Prior Work

• DragAPart significantly outperforms state-of-the-art methods on PSNR, SSIM, and LPIPS across the I.D. Drag-a-Move and Human3.6M evaluations.The comparison includes iPoke trained under the identical setting and DragNUWA, which primarily moves whole objects rather than modeling part-level interactions.Source: §5.2 · p. 11, Table 1 · p. 11
• Qualitatively, DragAPart preserves object identity while producing physically plausible part-level motion, unlike baselines that often reposition, rescale, or distort objects.On humans, it produces results closest to ground truth without changing actor identity; real-image results preserve textures, thin structures, shades, and multi-part motion.Source: §5.2 · p. 12, Figure 4 · p. 12

5.3 Ablation Analysis and Limitations

• Multi-resolution flow encoding injected into every LDM block improves performance across architectures, while input-only conditioning is insufficient.The encoding separates different drags and better enforces disentanglement among part motions; the qualitative ablations agree with the numerical results.Source: §5.3 · p. 13, Table 2 · p. 13, Figure 5 · p. 13
• Training with random textures improves out-of-distribution robustness and generalization to unseen categories, whereas removing it causes significant performance degradation.The randomization addresses synthetic textures that could let the model match pixel values without learning correspondence to the original image.Source: §5.3 · p. 13, Table 3 · p. 13, Figure 6 · p. 14
• The method does not explicitly enforce consistency across viewpoints and drag conditions and currently requires separate models for everyday objects and humans.The authors identify a unified model for all moving entities as a direction toward a universal motion prior.Source: §5.3 · p. 13

5.4 Application Results

• DragAPart supports motion analysis by recovering drag-specified motion parameters and coarse segmentation of the moving parts.The segmentation reflects the input drag, and the model shows relative consistency across viewpoints in the motion-analysis examples.Source: §5.4 · p. 14, Figure 7 · p. 14

6 Conclusion

• DragAPart uses drags as an interface for part-level dynamics and interprets them as fine-grained interactions with an object.The method uses a small amount of contributed synthetic data and domain randomization.Source: §6 · p. 14
• DragAPart obtains better results than other methods on the part-level dragging task, partly through its new drag encoder.Source: §6 · p. 14
• The model generalizes to real-world images and novel categories and supports applications of its learned motion prior.Source: §6 · p. 14

A.1 Implementation Details

• The implementation-details appendix extends Section 3.2 with further architectural components.Source: §A.1 · p. 19
• Drag encodings are patchified into tokens and converted into scale and shift parameters for each DiT layer.These parameters are added to those regressed from the diffusion time step.Source: §A.1 · p. 19
• Global image control uses CLIP ViT/L-14 features aggregated into image features that replace text or class conditioning.The features replace CLIP textual encoding in SD UNet or class labels in DiT.Source: §A.1 · p. 19

A.2 Experiment Details

• Training uses 4× NVIDIA A40 GPUs, batch size 4 × 32, learning rate 10^-5, AdamW, and 150,000 iterations.During the final 50,000 iterations, batches contain 20% random-texture renderings and 80% regular renderings; drags are zeroed 10% of the time.Source: §A.2 · p. 19
• Inference generates samples with 50 denoising steps and classifier-free guidance weight 5, taking roughly 2 seconds per image on one NVIDIA A40.Source: §A.2 · p. 19
• Quantitative metrics are computed at 256 × 256 after aspect-ratio adjustment, and baseline-specific textual prompts are supplied when required.DragNUWA outputs are padded, resized, then unpadded and resized for fair comparison; prompts are specified for text-conditioned baselines.Source: §A.2 · pp. 19–20

A.3 Drag-a-Move Dataset Details

• Drag-a-Move retains objects with at least one substantially moving part labeled Hinge Handle, Slider Drawer, Hinge Lid, or Hinge Door.Objects lacking these kinematic types are discarded because some annotated components exhibit minimal or no movement.Source: §A.3 · p. 20
• Each part receives an articulation state from fully closed to fully open, and animations sample moving subsets while treating parts as essentially independent.The construction uses 36-frame animations and two animation types selected with probability 0.5.Source: §A.3 · p. 20
• Type-1 animations move selected parts linearly from closed to open while unselected parts remain closed; type-2 animations use random stationary, starting, and ending states.The supplied formulation specifies these state transitions for moving and stationary parts.Source: §A.3 · p. 20
• Training randomly pairs a reference frame with a ground-truth frame to be noised, regardless of temporal order.Source: §A.3 · p. 20

A.4 Details on Drag-Driven Moving Part Segmentation

• For moving-part segmentation, the model adds noise to the input latent, encodes drags with the flow encoder, and extracts diffusion features for the image-drag pair.The drag-sensitive representation is formed by comparing UNet features with drags against features without drags.Source: §A.4 · pp. 20–21
• The segmentation pipeline builds a feature pyramid from self-attention outputs and applies K-means clustering, assigning pixels using clusters associated with drag tails.The diffusion time step and number of clusters are treated as hyperparameters.Source: §A.4 · p. 21

• Despite training exclusively on white-background images, DragAPart fills in occluded background details in complex real-world scenes.The authors attribute this capability largely to the strong priors of the pre-trained Stable Diffusion model.Source: §B.1 · p. 21

B.2 Failure Cases

• DragAPart occasionally misrecognizes object motion parts, distorts unseen categories unrealistically, or generates physically implausible images.The authors suggest that the limited scale of the training data may contribute to these limitations.Source: §B.2 · p. 21

B.3 Quantitative Comparisons of Moving Part Segmentation

• DragAPart features yield the highest moving-part segmentation mIoU and remain relatively robust across hyperparameter settings.The comparison clusters DragAPart features against CLIP, DINO, and Stable Diffusion features on the Drag-a-Move test split.Source: §B.2 · p. 22, §B.3 · p. 22
• Feature visualizations show that DragAPart representations are smooth and contain part-level information.The features are compared with CLIP, DINO, and Stable Diffusion representations across diffusion time steps for Stable Diffusion.Source: §B.2 · p. 22, §B.3 · p. 22