Parasitic Co-Denoising: Unlocking 3D Human Motion Generation in a Frozen Video Diffusion Model

Introduction

• Motion-capture scarcity limits text-to-motion coverage of long-tail behavior, while video diffusion learns varied human motion from internet-scale data without explicit 3D supervision.The paper asks whether this implicit knowledge can become explicit 3D motion generation without training a separate motion model.Source: Introduction · p. 1
• Video-guided motion methods require large paired video-motion datasets and a separate motion branch, motivating direct decoding from the video model’s generative process.The authors frame that separate pipeline as rebuilding capacity the video model may already possess, with added training and storage costs.Source: Introduction · p. 1
• Parasitic co-denoising decodes motion from the host along its denoising schedule, with the motion decoder sharing that schedule and conditioning each step on matched-σ host features.Video generation and motion decoding proceed in step.Source: Introduction · p. 1
• PMD is a flow-matching decoder attached to frozen Wan2.1 that fuses intermediate DiT features through a σ-adaptive mechanism and updates motion latents without modifying the host.The fusion learns which layers to attend to at each noise level.Source: Introduction · pp. 1–2
• Across σ, frozen Wan2.1 retrieves paired motion at 24–31% (chance 0.25%) and reaches 54–81% of HY-Motion accuracy, peaking at 81% at σ = 0.9.The input is nearly pure Gaussian noise at σ = 0.9.Source: Introduction · p. 1, Figure 1 · p. 2
• At a small fraction of dedicated generators’ trainable parameters, PMD leads them on text-motion alignment while producing paired video and motion in a single pass.The paper also reports that PMD inherits the video model’s broad semantic coverage and generalizes to complex actions outside motion-specific datasets’ reach.Source: Introduction · p. 2

Text-to-motion generation

• Text-to-motion methods include variational latents (Petrovich, Black, and Varol 2022; Guo et al. 2022b), sequence diffusion (Tevet et al. 2022), and token-based generation (Zhang et al. 2023a; Guo et al. 2024).Later work adds physical plausibility, fine-grained control, or language-model backbones.Source: Related Work · p. 2
• MotionLCM (Dai et al. 2024) accelerates inference via consistency distillation, while MotionCraft (Bian et al. 2025) scales motion training data.Source: Related Work · p. 2

Reusing pretrained generative models

• Reuse of pretrained diffusion features supports tasks including semantic segmentation (Baranchuk et al. 2021), semantic correspondence (Tang et al. 2023), monocular depth (Ke et al. 2024), and self-supervised representation learning (Xiang et al. 2023).The passage also cites diffusion-feature work on self-conditioning (Feng, Zhang, and Zhao 2024).Source: Related Work · p. 2
• Motion-domain reuse includes video-feature supervision in ViMoGen (Lin et al. 2025), jointly fine-tuned video and motion branches in EchoMotion (Yang et al. 2025), and subspace alignment in MoAlign (Bhowmik et al. 2025).PMD instead participates in denoising, decoding motion from intermediate states at every σ rather than using clean-endpoint readouts or distilling host weights into a separate model.Source: Related Work · p. 2

Stage 1: Motion-Video Latent Alignment

• Stage 1 maps motion and video representations into a shared latent space, using motion reconstruction, video-motion alignment, and latent-magnitude regularization to bridge their mismatch.The motion encoder and decoder form one pipeline, while a projection compresses video latents into the same space; the alignment is intended to avoid ill-conditioned flow-matching training.Source: Stage 1: Motion-Video Latent Alignment · p. 3

Stage 2: Parasitic Motion Decoder

• PMD denoises motion in lockstep with the video DiT on the same schedule, conditioning on host features at matched σ despite independent noise draws.The motion decoder predicts velocity at each σ, rather than learning an independent flow.Source: Stage 2: Parasitic Motion Decoder · pp. 3–4
• Selected intermediate DiT features are mapped into the shared latent space, adapted per layer, and fused with σ-dependent weights learned from the timestep feature.The host output head and Stage 1 projection are reused; only per-layer adapters and the fusion MLP are added in this stage.Source: σ-Adaptive Multi-Layer Fusion · p. 4, Stage 2: Parasitic Motion Decoder · p. 4
• PMD blocks combine temporal self-attention, video and text cross-attention, and feed-forward processing, with each sub-module conditioned on σ.The host DiT, video VAE, text encoder, and Stage 1 modules are frozen; at inference, the host and PMD advance jointly and one pass produces video and aligned motion.Source: σ-Adaptive Multi-Layer Fusion · p. 4

Quantitative Results

• PMD leads both MBench semantic metrics under both protocols, outperforming dedicated generators in Protocol A and larger ViMoGen and MotionVLA models in Protocol B.The advantage spans scarce to abundant data and comes with a small fraction of their trainable parameters.Source: Table 1 · p. 5, Quantitative Results · p. 6
• On HumanML3D, PMD leads on R-Precision but trails the best MoCap-specialized methods on FID.The passage attributes this trade-off to inheriting the video host’s distribution rather than fitting HumanML3D’s curated indoor MoCap statistics.Source: Table 2 · p. 5, Quantitative Results · p. 6

Setup

• The probe pairs HumanML3D motions with appearance-randomized videos and tests host-motion alignment by retrieval across 400 held-out candidates and ten viewpoints.The reported recoverable signal spans the denoising schedule.Source: Setup · p. 5

Quantitative Results

• Across four MBench prompts, PMD follows prompts more faithfully and produces more expressive movement; baselines fall back on generic motion for underrepresented actions such as Salsa.The comparison also shows paired video alongside PMD’s motion, which the motion-only baselines cannot produce.Source: Quantitative Results · p. 6, Figure 3 · p. 6

Discussion

• Paired video and motion latents move in the same direction at each σ, while eight seeds produce distinct, host-consistent motions per video (40 ± 12 mm within-video pairwise MPJPE).Centered cosine similarity peaks near σ ≈ 0.5 and remains high toward zero.Source: Figure 5 · p. 7, Discussion · p. 7, Figure 6 · p. 7

Ablation

• Removing the host signal collapses PMD toward a same-size text-only generator; removing text loses fine prompt control, while removing Stage 1 alignment collapses training.The ablations distinguish the host’s semantic contribution from text conditioning and alignment’s role in making host features usable as motion.Source: Ablation · p. 7
• A σ-adaptive fusion outperforms final-layer, single-L29, and static-uniform alternatives; L29 dominates mid-late, while early- and late-layer contributions shift across denoising.The learned weights indicate that both layer choice and changing mixtures matter.Source: Figure 4 · p. 6, Ablation · p. 7
• Hooks spread across the host’s depth outperform shallow hooks, while three full-depth layers recover most of the gain.The result points to depth coverage rather than layer count as the key condition.Source: Ablation · p. 7

• At a small fraction of dedicated motion generators’ trainable parameters, PMD leads them on text-motion alignment.Source: Conclusion · p. 7
• Parasitic co-denoising decodes 3D human motion from a frozen video diffusion model along its denoising schedule.Source: Conclusion · p. 7
• PMD attaches a flow-matching decoder to a frozen video host through σ-adaptive multi-layer fusion.Source: Conclusion · p. 7
• The authors hope the paradigm extends to other modalities whose generation is implicit in larger generative processes.Source: Conclusion · p. 7