Source-linked AI summary
Motion4Motion: Motion Transfer Across Subjects at Inference
Ling-Hao Chen, Zixin Yin, Duomin Wang, Xianfang Zeng, Gang Yu
TL;DR
Motion transfer methods struggle to generalize beyond human-like characters because they rely on predefined skeletons and scarce paired data. Motion4Motion instead transfers pixel-level motion flows during inference without training, outperforming state-of-the-art baselines in motion fidelity and appearance preservation.
Problem
Existing motion-transfer methods rely on structural priors and scarce paired data, limiting robust transfer across species with different morphologies and motion styles.
Method
Motion4Motion is a training-free framework that tracks dense pixel-level motion flows and injects them into attention through TransPE without skeletal correspondence.
Results
Motion4Motion consistently achieves the best performance across evaluated metrics and significantly outperforms state-of-the-art baselines in motion fidelity and appearance preservation.
Takeaways & Limitations
The framework supports high-fidelity motion transfer across diverse subjects, including inanimate objects, without skeletal priors or specific fine-tuning.
Abstract
from arXiv · showhide
This work explores the motion transfer from one video to another, which is crucial in animation for diverse characters. Previously, video motion transfer has been largely explored between human and human-like characters, enabling a lot of applications in digital creation. However, these approaches encounter a main limitation. Specifically, related technical pipelines heavily rely on a predefined human skeleton structure and accordingly require skeleton-conditional model training. On the one hand, these methods are difficult to generalize to diverse characters, such as animals from different species, while preserving their unique motion styles. On the other hand, labeled data in diverse skeletons is limited, which additionally restricts the large-scale training for the task. In this paper, we jump out of the skeleton-based motion transfer framework and propose a training-free motion transfer framework, named Motion4Motion. Motion4Motionmodels the motion flow of the character in a video instead of skeletons, which makes motion transfer across species easier. Extensive experimental results and novel applications show our methods outperform baselines impressively. Project page is available at https://lhchen.top/Motion4Motion.
1 Introduction
The introduction identifies skeletal priors and scarce paired data as barriers to motion transfer beyond human-centric subjects. Motion4Motion addresses these limitations with a training-free, pixel-level motion-flow framework and TransPE module for cross-subject transfer.
- Motivation: Motion transfer supports digital creation and animation workflows, but mainstream research primarily migrates movements between human subjects using skeletal representations.Applications include character animation, virtual reality, and movie post-production.
- Challenges: Hard-coded structural priors restrict motion transfer across species because characters with different morphologies lack a shared skeletal template.The passage identifies this limitation as especially problematic beyond human-centric domains.
- Challenges: Scarcity of high-quality paired motion data across diverse topologies impedes robust cross-species motion transfer.Synchronized motion sequences for diverse characters are described as labor-intensive and often impractical to obtain.
- Method: Motion4Motion departs from skeleton-based pipelines by representing motion with dense pixel-level motion flows and transferring them to target subjects through TransPE.The framework captures source-pixel temporal dynamics and maps them onto the target subject without structural limitations.
- Contributions: Motion4Motion is training-free and operates entirely during inference, enabling motion transfer across general subjects without predefined skeletal priors.The contributions also report high-fidelity transfer and potential applications in novel concept composition and cross-morphology motion transfer.
2 Related Work
Related work spans diffusion architectures, attention-based controllable generation, and motion transfer in both 3D and video. Existing motion-transfer methods use skeletal or topology-specific guidance, motivating approaches that generalize beyond fixed character structures.
- Diffusion Models: Diffusion Transformers emerged as a scalable backbone for state-of-the-art video diffusion models after early U-Net diffusion systems surpassed GANs in image fidelity.The passage attributes the shift to scalability constraints in U-Net-based diffusion models.
- Controllable Generation: Attention-control methods modulate pre-trained U-Nets for image and video editing, including key-value injection and concatenation within attention layers.The field originated with Prompt-to-Prompt and expanded to multiple controllable-generation applications.
- 3D-based motion transfer: 3D motion transfer originated as motion retargeting and evolved from kinematic optimization toward neural methods that decouple pose from character structure.Traditional approaches enforced spatial constraints, whereas neural approaches use deep architectures for pose–structure decoupling.
- Video-based motion transfer: Video motion-transfer frameworks rely on large-scale pre-training and explicit skeletal guidance, restricting applicability to fixed topologies and requiring substantial computation.The passage contrasts specialized human-animation systems with more general motion-editing approaches.
3 Methodology
Motion4Motion is a training-free framework that transfers motion across subjects by manipulating denoising attention rather than relying on skeleton correspondence. It establishes semantic point correspondences, extracts source motion flow, maps that flow to the target, and injects position-aware features into attention to guide generation.
- Framework Overview: Motion4Motion manipulates denoising attention in a training-free framework, tracking subjects’ motion flow instead of using skeleton correspondence.This design supports motion transfer across subjects without predefined skeleton structures.
- Cross-image correspondence: The method establishes skeleton-free cross-image correspondence by semantically matching sampled source anchor points to corresponding coordinates on the target subject.Grounded SAM-2 samples representative source points, while semantic matching finds their target coordinates.
- Motion flow extraction: Motion flow is extracted by tracking source anchor-point trajectories across subsequent frames with DIFT features, producing a topology-agnostic representation of the source dynamics.The resulting flow is later injected into the target generation process through attention manipulation.
- Target motion flow construction: The target motion flow is constructed by composing cross-image correspondence with source-video tracking, mapping target semantic points to their intended spatio-temporal destinations.The formulation derives M = I ◦C−1 and aims to reproject target points so that M_tgt = M_src.
- Attention manipulation: Position-aware features are padded into attention space to force denoising to synthesize the target subject at coordinates dictated by the motion flow, without tuning.This attention manipulation is described as achieving high-fidelity motion transfer.
4 Experiments · 4.1 Setting
Motion4Motion is evaluated as a training-free, attention-manipulation framework built on WAN-14B-T2V, using motion-flow extraction and diffusion-based feature matching rather than predefined pose representations. Experiments cover animal and human transfer benchmarks, plus a new cross-category animal pose-similarity benchmark.
- 4.1.1 Implementation details.: Motion4Motion is implemented on WAN-14B-T2V at 480p, applying attention manipulation across layers [0, 40] through denoising step 35 of 50.TransPE is applied in all self-attention layers within this range.
- 4.1.1 Implementation details.: Point matching uses diffusion feature matching, while subject segmentation uses SAM-2 to align masks and point coordinates with latent-space resolution.The passage states that both masks and coordinates are aligned with the latent space, although the remaining implementation description is truncated.
- 4.1.1 Implementation details.: Existing motion-transfer methods typically target humans or human-like characters and rely on predefined global motion signals or explicit pose representations.WAN-animate is cited as heavily depending on human pose detectors, whose performance may be unstable for non-human-like characters.
- 4.1.2 Baselines.: The evaluation adopts FlexAct and Motion4Motion baselines to assess fine-grained character motion control, while MotionV2V and MotionShot were unavailable before submission.The supplied passage names FlexAct and begins naming another state-of-the-art method, but that name is truncated.
- 4.1.2 Baselines.: Algorithm 1 extracts source motion flow by sampling anchor points, matching them semantically to the target image, and tracking them through the source video.The algorithm labels these operations as Stage 1: Motion Flow Extraction.
- 4.1.2 Baselines.: The subsequent pipeline initializes target latents from inversion and performs denoising with TransPE manipulation.The algorithm labels these operations as Stage 2: Latent Initialization and Stage 3: Denoising with TransPE.
- 4.1.2 Baselines.: Motion4Motion is evaluated on animal motion transfer with 33 pairs and human motion transfer with 123 pairs from benchmarks introduced by Zhang et al.Because the framework uses WAN-T2V-14B, evaluation first deterministically inverts the source video to obtain a base latent path, then applies attention manipulation.
- 4.1.3 Benchmark and evaluation protocol.: A new benchmark contains 50 image-video pairs with different animals and evaluates fine-grained pose alignment using pose similarity between source and target motion.The examples are covered by a cross-category pose detector; identity preservation is evaluated using average CLIP similarity between the target image and generated video frames.
4.2 Evaluation
Motion4Motion outperforms state-of-the-art baselines across all reported metrics and provides finer-grained motion control. Visual comparisons show stronger cross-species motion fidelity, pose alignment, appearance consistency, and training-free generalization to in-the-wild subjects.
- Quantitative comparison: Motion4Motion achieves the best performance across all metrics in comparison with several state-of-the-art baselines.The paper attributes this superiority to its fine-grained point-to-pixel control mechanism.
- Quantitative comparison: Point-to-pixel control enables Motion4Motion to capture local motion nuances beyond global representations or model-level tuning.The comparison specifically contrasts Motion4Motion with MotionDirector and RoPECraft.
- Visual comparison: Existing baselines struggle to balance motion fidelity and visual quality in challenging cross-species transfers, while RoPECraft produces stiff movements and misses fluid leg dynamics.The examples include transferring a fox’s gait to a giraffe and a lion’s pounce to a zebra.
- Visual comparison: Motion4Motion maintains high-fidelity pose alignment and appearance consistency, whereas most baselines exhibit visual artifacts or appearance drifting.The comparison focuses on preserving pose details across individual frames.
- In-the-wild generalization: Without any training, Motion4Motion successfully retargets complex dance motion to static portraits of diverse subjects from in-the-wild Internet images and videos.The examples include public figures with distinct stylistic features.
4.3 User Study
A blind pairwise user study evaluated generated videos across motion and appearance consistency. Motion4Motion significantly outperformed the baseline on both criteria, indicating faithful motion transfer and stronger visual fidelity.
- Evaluation Protocol: The study used blind pairwise comparisons between Base Model (WAN-I2V-14B) outputs and competing methods or ours across 50 randomly selected test cases.All 10 raters selected their preferred result for each case.
- Evaluation Criteria: Participants assessed Motion Consistency and Appearance Consistency, measuring transferred-motion accuracy and preservation of the subject’s visual identity.Motion Consistency concerns alignment with source motion, while Appearance Consistency concerns visual identity preservation.
- Results: Motion4Motion demonstrated a significant advantage over the baseline on both motion and appearance consistency.The results support faithful motion transfer and higher visual fidelity compared to baselines.
5 Application: Teaching a Table Walking
This application targets the difficult composition of a desk running by transferring motion from a walking human to a static table. Motion4Motion uses SAM2 masks to bind corresponding legs and constrain diffusion feature matching, mitigating structural ambiguity from their morphological differences.
- Challenge: Large video generation models struggle to compose the novel concept of a desk running, instead producing static desk motions or sliding results.The mismatch between “running” and “desk” makes reasonable synthesis difficult.
- Cross-morphology transfer: Motion4Motion drives a static table using a video of a walking human, demonstrating cross-morphology motion transfer.The application uses Fig. 8B as the target table and Fig. 8A as the human motion source.
- Mask-based control: SAM2 masks explicitly bind the human legs to the table’s legs, constraining diffusion feature matching within the masked regions.This design is inspired by bone binding in 3D animation.
- Mask-based control: The masked feature matching mitigates structural ambiguity caused by morphological disparities and successfully renders the transfer tractable.The application addresses the significant challenge created by distinct structural differences between humans and tables.
6 Conclusion and Discussion
Motion4Motion is a training-free framework for high-fidelity motion transfer across diverse subjects without skeletal priors or subject-specific fine-tuning. Its TransPE module injects source-video motion flows into a pre-trained Diffusion Transformer’s self-attention, enabling semantic alignment across disparate morphologies.
- Conclusion and Discussion: Motion4Motion achieves high-fidelity motion transfer across diverse subjects without relying on skeletal priors or specific fine-tuning.The framework is explicitly training-free.
- Conclusion and Discussion: TransPE injects motion flows from a source video into the self-attention mechanism of a pre-trained Diffusion Transformer.This mechanism supports motion transfer without skeletal conditioning.
- Conclusion and Discussion: The approach enables precise semantic alignment even across subjects with disparate morphologies.The supplied passage attributes this capability to the proposed motion-flow injection mechanism.
- Conclusion and Discussion: Extensive evaluations demonstrate the effectiveness of Motion4Motion.The supplied passage introduces this evaluation result but ends before specifying quantitative findings.