WeLike2Party! In-Context Motion Transfer for Multi-Human Image Animation

• Existing methods condition video generation on explicit 2D skeletons or 3D meshes, whose estimation errors and inter-person occlusion can cause artifacts, identity switches, and incorrect motion binding.Prior approaches include Xu et al., 2024b; Hu, 2024; Zhang et al., 2024; Zhu et al., 2024; Wang et al., 2025a; Tu et al., 2025.Source: §1 · p. 2
• WL2P concatenates reference-image, driving-video, and noisy-target visual representations into one sequence, jointly using identity, motion, and interaction cues without explicit intermediate representations.Naive in-context conditioning can lose per-subject appearance detail and lacks explicit motion-binding supervision.Source: §1 · p. 2
• RARC preserves fine-grained appearance through higher-resolution reference encoding and spatial RoPE rescaling, while IBS supervises target-to-reference attention with per-subject masks to bind identities to motion trajectories.RARC rescales reference spatial coordinates to the target grid’s range before RoPE; IBS uses ground-truth instance masks.Source: §1 · pp. 2–3
• MotionTwin provides 14.4K cross-identity video pairs across 84.3 hours of photorealistic 1080p video, and MotionTwin-Bench contains 300 pairs for evaluating subject fidelity and identity-motion binding.Generated in Unreal Engine 5 by retargeting shared motions across identities, MotionTwin draws on 12.4K motion sequences, 4.5K human-asset combinations, and 357 HDRI environments, with filtering for implausible interactions.Source: §1 · p. 2
• WL2P improves subject-level fidelity, identity-motion binding, and overall perceptual quality over recent state-of-the-art methods, including in heavily occluded multi-person interactions.The paper reports evaluation on MotionTwin-Bench and diverse real-world inputs.Source: §1 · pp. 2–3

Controllable video diffusion models

• In-context video conditioning avoids compact explicit-representation bottlenecks, but rich visual context alone does not resolve correspondence between identities and motions in multi-human animation.Earlier controllable diffusion methods also use structural signals, kinematic priors, reference appearance, and camera controls.Source: §2 · p. 3

Human image animation

• Human animation commonly separates explicit driving-motion representations from reference appearance, but can lose geometry and interaction cues, inherit estimator errors, and struggle with overlapping or exchanging subjects.Multi-character methods add instance-aware conditions and correspondence mechanisms, while recent video-conditioned methods preserve richer cues but inherit synthetic-data errors; SCAIL-2 needs inference-time masks, and DreamActor-M2 and Wan-Animate 2 lack explicit identity binding.Source: §2 · p. 3
• Cross-identity pairs retarget shared motion and camera trajectories to different identities, providing ground-truth correspondence without pose-driven synthesis; instance masks supervise identity binding only during training.Pairs cover up to seven subjects; the MotionTwin pipeline filters SMPL-X motions, samples avatar appearances, and renders shared-motion pairs with reference images and instance maps.Source: §2 · p. 3, Figure 2 · p. 4

3 DATASET: MOTIONTWIN

• Each training pair combines an identity reference image, a driving video, and a target video with shared pose sequence and camera trajectory but different subjects and scenes.Because precisely synchronized real-world cross-identity data is nearly impossible to collect, MotionTwin is rendered synthetically using Unreal Engine 5 (Epic Games, 2026).Source: §3 · p. 4
• Motion sequences from public datasets are unified as SMPL-X and filtered for low activity, rotation glitches, excessive travel, repeated choreography, and severe inter-person mesh penetration.The filtering thresholds include less than 10−3 radians per frame for static segments, rotations over 60◦ per frame, root excursions beyond 8 m, and penetration deeper than 4 cm; shallow natural contacts are retained.Source: §3 · p. 4
• Each filtered sequence is rendered in two appearance variants with different identities, clothing, hairstyles, lighting, and backgrounds, with perturbed body shapes to teach motion correspondence across body proportions.Scene environments are sampled from 357 HDRIs.Source: §3 · p. 4
• Single-person reference images are sampled from target videos, while multi-person references place subjects side by side in first-frame order to handle occlusion; posed-geometry instance masks provide subject-level correspondence.The instance masks are generated in Blender (Blender Online Community, 2023) for identity-binding supervision in Sec. 4.4.Source: §3 · p. 4
• 14.4K cross-identity pairs comprise 8.3K single-person and 6.1K multi-person pairs, totaling 28.9K photorealistic videos and 84.3 hours at 1920 × 1080.Multi-person pairs cover two to seven interacting subjects and include rest-pose reference images and per-person instance maps; the dataset provides 12.3K reference images with ground-truth identity correspondence.Source: §3 · p. 5

4 PROPOSED METHOD: WL2P

• The framework introduces RARC to improve fine-grained appearance representation and IBS to associate reference identities with their intended motion.The method section also introduces the DiT backbone and in-context conditioning layout.Source: §4 · p. 5

4.1 PRELIMINARY

• WL2P builds on Wan2.1-I2V-14B, encoding video into latent representations and training its DiT to predict target velocity under a flow-matching objective.The forward process adds Gaussian noise to the clean latent.Source: §4.1 · pp. 5–6
• The backbone’s 3D RoPE encodes video tokens’ temporal and spatial positions by applying the same rotations to query and key features, allowing attention to capture relative distances while preserving content.Source: §4.1 · p. 6

4.2 IN-CONTEXT CONDITIONING

• WL2P uses in-context conditioning to concatenate clean reference-image and driving-video tokens with noisy target-video tokens, enabling direct self-attention between conditions and targets without architectural modification.The segments are independently VAE-encoded and patchified before sequence concatenation.Source: §4.2 · p. 6

4.3 RARC: REFERENCE ASYMMETRIC ROPE CONDITIONING

• RARC assigns fractional spatial RoPE coordinates to higher-resolution reference tokens, keeping them within the target tokens’ spatial range to preserve fine-grained appearance details.Directly extending the reference grid would place tokens outside the pretrained coordinate range, potentially disrupting spatial understanding and producing repetitive or implausible content.Source: §4.3 · p. 7
• By scaling reference-token positions into the target grid’s RoPE frame, RARC gives self-attention spatial cues for transferring appearance details without requiring strict reference-to-target spatial correspondence.The relative displacement is scaled by the target-to-reference grid dimensions.Source: §4.3 · p. 7

4.4 IBS: IDENTITY BINDING SUPERVISION

• IBS addresses identity swaps and drift by supervising reference-to-target attention with MotionTwin instance masks, enforcing subject-level identity association without pixel-level alignment.It normalizes attention over reference tokens to control allocation among identities, applies supervision at high-noise timesteps, and adds no inference parameters or cost.Source: §4.4 · p. 7

5.3 COMPARISON WITH STATE-OF-THE-ART METHODS

• WL2P achieves the best benchmark results across reconstruction fidelity and identity-motion binding, with stronger masked metrics and IAAcross under inter-person occlusion.MultiAnimate has the strongest baseline image reconstruction results, while SCAIL has the highest baseline binding accuracy.Source: Table 1 · p. 8, §5.3 · p. 8

5.1 IMPLEMENTATION DETAILS

• Training updates only Wan2.1-I2V-14B self-attention blocks; an 81-frame 480p inference clip takes 6.5 minutes on one B200 GPU.IBS uses γ = 0.7 and β = 0.05 and is restricted to the top 40% of the noise range.Source: §5.1 · p. 8

5.2 EVALUATION PROTOCOL

• The evaluation uses 300 cross-identity MotionTwin-Bench pairs and 161 internet-collected real-world image-video pairs, measuring video and subject-level fidelity, identity assignment, and perceptual quality.IAA scores whether all frame-level identity assignments are correct; IAAcross restricts evaluation to frames with inter-person occlusion.Source: §5.2 · p. 8

5.3 COMPARISON WITH STATE-OF-THE-ART METHODS

• WL2P leads in video, frame, and I2V quality on real-world inputs and earns the highest user-study preference scores for identity preservation, binding, and visual quality.The user study involved 20 participants evaluating 20 real-world clips against four baselines.Source: §5.3 · pp. 8, 10, Table 2 · p. 9
• In qualitative comparisons, WL2P preserves reference-subject appearance and intended motion associations, while baselines show appearance degradation, identity bleeding, drift, swaps, or subject duplication.The comparisons cover single- and multi-person animation, including challenging interactions and occlusion.Source: Figure 4 · p. 9, §5.3 · pp. 9–10

5.4 ABLATION STUDIES

• Ablations show that RARC improves subject fidelity over standard RoPE with high-resolution references, while IBS improves IAA and IAAcross over the baseline.The vanilla in-context baseline already has competitive subject fidelity and identity-motion binding; standard RoPE performs comparably to or worse than baseline, which the authors attribute to positional distribution shifts.Source: §5.4 · p. 10, Table 3 · p. 10

6 CONCLUSION

• WeLike2Party transfers motion directly from driving videos without explicit pose or mesh estimation, combining appearance preservation and identity-motion binding with cross-identity training and evaluation resources.RARC uses high-resolution reference encoding and fractional spatial RoPE; IBS applies an auxiliary target-to-reference attention loss with per-subject instance masks. MotionTwin supports paired cross-identity training, while MotionTwin-Bench evaluates subject-level visual fidelity and identity-motion binding using ground-truth targets and correspondences.Source: §6 · p. 10
• Fine-grained identity details may degrade as more subjects share a fixed-resolution frame, and computational expense limits interactive animation.Suggested directions include subject-specific face embeddings, efficient attention or distillation, and MotionTwin supervision with independently controlled camera trajectories and subject motions.Source: §6 · p. 10

AI USE STATEMENT

• Generative AI tools were used only to polish manuscript readability and grammar; authors reviewed the edits and take responsibility for the final content.The tools were not used to develop the methodology, implement the method, or interpret experimental results.Source: AI USE STATEMENT · p. 11

REPRODUCIBILITY STATEMENT

• The paper provides detailed descriptions of the architecture, training objectives, and MotionTwin construction pipeline; code, checkpoints, and benchmark data are slated for release after the final decision.Source: REPRODUCIBILITY STATEMENT · p. 11

Data distribution

• MotionTwin contains 14,427 motion clips, each rendered with two identities, totaling 84.3 hours; multi-person clips account for 42.7% of clips and 35.2% of hours.The motions come from eight public corpora: AMASS, Inter-X, DD100, CoMPaS3D, CHI3D, Harmony4D, Hi4D, and GDance. Every clip includes per-subject instance maps, and multi-person clips also include rendered identity-reference images.Source: A ADDITIONAL DETAILS ON MOTIONTWIN… · p. 17

Motion preprocessing

• Source motions are converted to SMPL-X sequences at 30 fps, filtered for artifacts and unsuitable segments, and multi-person subjects are jointly centered to preserve interaction geometry.Retained sub-windows keep horizontal root trajectories within 8 m; clips with severe inter-person mesh penetration are rejected.Source: A ADDITIONAL DETAILS ON MOTIONTWIN… · p. 17

Rendering pipeline

• Paired videos share motion, camera trajectory, and temporal length but differ in identity and appearance; masks are rasterized from posed SMPL-X geometry, with multi-person identity references rendered in canonical standing poses.Single-person cameras use near-frontal views that track the subject, while multi-person clips use a static camera capturing the group.Source: A ADDITIONAL DETAILS ON MOTIONTWIN… · p. 17

Assets

• Appearance variants use distinct BEDLAM and BEDLAM 2.0 identity assets, with perturbed SMPL-X shape coefficients and scenes sampled from 357 Poly Haven HDRI maps.The assets vary body shape, garments, hairstyles, and skin appearance; scenes cover indoor and outdoor environments.Source: A ADDITIONAL DETAILS ON MOTIONTWIN… · p. 17

B ADDITIONAL DETAILS ON TRAINING WL2P

• Training updates self-attention projections in all 40 Wan2.1-I2V-14B blocks: 4.20B of 16.39B parameters (25.6%).The updated projections are query, key, value, and output.Source: B ADDITIONAL DETAILS ON TRAINING WL2P · p. 19
• Optimization uses AdamW at a constant 1 × 10−5 learning rate, four B200 GPUs, effective batch size 4, bf16, and ZeRO stage 2.Training uses weight decay 0.01, no warmup, gradient clipping at 1.0, and no gradient accumulation. One fixed text prompt is used for all training samples and inference; classifier-free guidance (scale 5.0) applies only to text, with image and video conditions shared across branches.Source: B ADDITIONAL DETAILS ON TRAINING WL2P · p. 19
• IBS supervises attention between labeled reference-image and target tokens only when at least two subjects are labeled in both.A token is labeled when one subject covers at least 70% of its 16 × 16 region; backgrounds, boundaries, and occluded areas remain unlabeled. After RoPE, per-head query-key loss uses up to 256 target tokens per subject and all labeled reference tokens, averaged over heads and blocks; it is weighted by β = 0.05 at timesteps τ ≥0.6. IBS adds no parameters and leaves inference unchanged.Source: B ADDITIONAL DETAILS ON TRAINING WL2P · p. 19

Benchmark construction

• MotionTwin-Bench uses held-out clips with large motion, frequent turning, and close interactions, excluding sequences with severe mesh interpenetration.It follows the training set’s rendering and appearance-generation pipeline, and provides per-subject instance maps for identity correspondence.Source: C ADDITIONAL DETAILS ON MOTIONTWIN-BENCH · p. 19

Reference images

• Reference images use the target video’s camera, HDRI, and lighting, with subjects in initial ordering and canonical camera-facing standing poses.Reference images and instance maps are generated with the same geometry-based procedure as the target videos.Source: C ADDITIONAL DETAILS ON MOTIONTWIN-BENCH · p. 19

Identity assignment metrics

• IAA measures the fraction of frames where all subjects match their intended reference identities; IAAcross averages only frames with at least two overlapping subject boxes (IoU > 0.1).The overlap-restricted metric targets frames with inter-person proximity or occlusion.Source: C ADDITIONAL DETAILS ON MOTIONTWIN-BENCH · p. 19

Assignment procedure

• Identity assignment compares normalized 27-dimensional appearance signatures within ground-truth subject regions and selects the one-to-one mapping with minimum total L2 distance.Signatures comprise mean CIELAB color and 8-bin histograms for each RGB channel; assignment is exhaustive for P ≤4, and a frame is correct only when the mapping is the identity. Ground-truth regions make the procedure insensitive to pose and small misalignments.Source: C ADDITIONAL DETAILS ON MOTIONTWIN-BENCH · p. 19

Quantitative evaluation

• Table 4 reports single-person animation evaluation on 75 MotionTwin-Bench windows and 36 single-subject real-world clips using VBench metrics.The supplied caption does not provide metric values or establish a method winner.Source: Table 4 · p. 20

Single-person evaluation

• WL2P achieves the best reconstruction fidelity and FVD on MotionTwin-Bench single-person clips, and the best real-world video and frame quality while remaining competitive in I2V quality.Subject fidelity and identity binding are not reported because those metrics target multi-person scenarios; the evaluation includes single-person-specific methods.Source: D ADDITIONAL EXPERIMENTS · p. 20

Ablation study on RARC

• RARC maps high-resolution reference tokens into the target RoPE range, preserving distinct fingers and face appearance where native integer coordinates cause merged fingers and identity drift.The ablation holds reference resolution and driving video constant, showing that enlarging the reference alone does not recover fine details because extra tokens exceed the spatial range seen during pretraining.Source: D ADDITIONAL EXPERIMENTS · p. 20, Figure 7 · p. 21

• Figures 8–10 show additional qualitative results, labeling reference images, driving videos, MultiAnimate, Wan-Animate 2, Scail-2, and Ours.Source: Figure 8 · p. 22, E ADDITIONAL QUALITATIVE RESULTS · pp. 23–24, Figure 9 · p. 23