RASA: Disentangled Spatial-Motional Priors for Cross-Identity Character Animation

Zhen Xiao; Zhen Shen; Zhaofan Qiu; Ting Yao; Xueliang Liu; Tao Mei

Source-linked AI summary

RASA: Disentangled Spatial-Motional Priors for Cross-Identity Character Animation

Zhen Xiao, Zhen Shen, Zhaofan Qiu, Ting Yao, Xueliang Liu, Tao Mei

arXiv:2608.28219v1Aug 28, 2026cs.CV

TL;DR

Cross-identity character animation remains difficult because reference and driving characters can differ in scale, position, and skeletal proportions. RASA separates spatial calibration from semantic motion guidance using SPC and IMG within a DiT, and reports superior robustness over existing methods. Its scope is limited by 1:1 body-part correspondence and pose-based hand and facial motion.

Problem
Cross-identity animation remains challenging because reference images and driving poses can differ in scale, position, and skeletal proportions.
Method
RASA disentangles spatial mapping from motion control using SPC for spatial calibration and IMG for identity-agnostic semantic motion guidance in a DiT.
Results
RASA significantly outperforms state-of-the-art methods and shows superior robustness in cross-identity settings.
Takeaways & Limitations
Disentangled spatial and motional priors support robust character animation by separately addressing geometric calibration and semantic motion preservation.
Takeaways & Limitations
RASA assumes 1:1 body-part correspondence and focuses on full-body animation, while hand and facial motion quality is constrained by estimated 2D poses.

Abstract

from arXiv · show

Cross-identity character animation aims to drive a target identity from a reference image to follow the motion of a source character from a driving video. The core challenge lies in the inherent entanglement of two capabilities: cross-identity spatial mapping (aligning position, scale, and skeletal proportions) and motion control (refining joint articulation, volumetric consistency, and view coherence). We introduce Reference-Aware Structural Alignment (RASA), a framework that disentangles spatial mapping from motion control by injecting structured priors into a Diffusion Transformer (DiT). Our approach has two stages. First, a Spatial Prior Calibrator (SPC) fuses reference identity with driving pose to generate a spatially grounded initial noise latent, ensuring correct positioning, scaling, and alignment with the driving skeleton. Second, an Inherent Motional Guider (IMG) encodes shape-agnostic SMPL articulation parameters into a semantic motion vector beyond appearance-biased 2D keypoints. Injected into intermediate DiT layers, this vector complements the base pose condition for anatomically consistent articulation and view-aware volumetric refinement. We curate CIM-Bench, a high-quality benchmark with rigorous curation, for evaluation. Extensive experiments show RASA significantly outperforms state-of-the-art methods in motion fidelity and visual quality. Our work establishes a new paradigm showing disentangled spatial and motional priors are key to robust character animation. Project page: https://hidream.ai.github.io/RASA/

1 Introduction

Cross-identity animation remains difficult because spatial misalignment between reference characters and driving poses entangles geometric calibration with motion transfer. RASA addresses this by separating spatial and motional priors within a DiT-based framework and evaluates robustness with CIM-Bench.

Challenge: Cross-identity animation suffers from discrepancies in scale, global position, and skeletal proportions between reference characters and driving subjects.These pose–reference mismatches challenge high-fidelity motion transmission.
Challenge: Existing remedies, including pose retargeting, augmentation, and late-stage feature fusion, can lose information or entangle calibration with appearance rendering.The reported consequences include visual artifacts and unstable motion transfer.
Approach: RASA proposes disentangled spatial and motional priors to address pose–reference misalignment in cross-identity animation.Its central insight is to decouple spatial mapping from fine-grained motion execution.
Approach: The dual-injection design uses SPC for continuous 2D spatial calibration and IMG for intermediate identity-agnostic 3D semantic guidance.Together, the modules target geometric alignment and anatomically grounded motion guidance within the DiT backbone.
Evaluation: CIM-Bench provides a curated benchmark for evaluating motion fidelity under realistic structural misalignment.The benchmark addresses the lack of standardized protocols for misaligned cross-identity scenarios.

2 Related Work

Diffusion models and structured conditioning have advanced pose-driven character animation, but most methods still assume structural compatibility between driving poses and reference identities. RASA instead combines spatial calibration with anatomically consistent semantic guidance.

Diffusion generation: Diffusion models became dominant for video generation because of stable optimization and strong visual fidelity.Recent systems extend text-to-image diffusion models into the temporal domain using temporal modules or DiT backbones.
Structured conditioning: Structured conditions such as depth, sketches, human poses, and camera viewpoints improve controllability in diffusion pipelines.ControlNet and ControlNeXt are examples of condition-specific or lightweight conditional modulation designs.
Pose-driven animation: Pose-driven personalized generation transfers motion to a reference image while preserving character appearance.Recent diffusion-based approaches improve visual realism and motion fidelity for this task.
Structural misalignment: Most existing methods assume structural compatibility, so raw pose–appearance binding can distort geometry when scale or body proportions differ.Prior mitigation strategies include coarse normalization and SMPL-based parametric alignment.
RASA: RASA introduces SPC for 2D geometric calibration and IMG for anatomically consistent 3D semantic guidance within intermediate transformer layers.This contrasts with heuristic preprocessing and late-stage feature fusion.

3 Method

RASA uses hierarchical conditioning in a DiT-based flow-matching pipeline: SPC calibrates spatial structure, while IMG injects semantic motion guidance for anatomically plausible and viewpoint-consistent animation. CIM-Bench supplies realistic cross-identity evaluation data.

3.1 Preliminaries: Flow Matching interpolates between encoded video latents and Gaussian noise, then trains a conditioned network to predict the corresponding velocity field.Inference starts from Gaussian noise and recovers the clean latent by solving an ODE.
3.2 Overall Framework: RASA’s DiT backbone processes spatio-temporal latent patches with transformer blocks and 2D RoPE for spatial-temporal relationships.The backbone receives SPC-calibrated latents and intermediate 3D semantic guidance.
3.2 Overall Framework: Reference identity is encoded by a pretrained VAE and concatenated with noisy video latents, while positional offsets distinguish static appearance from temporal evolution.The DiT can attend globally to reference appearance while maintaining identity–motion separation.
Spatial Prior Calibrator: SPC learns structural calibration from perturbed driving poses and reference poses, adding aligned features to noisy latents at every denoising step.Training perturbations include stochastic scaling, translation, and proportional skeletal dropout.
Inherent Motional Guider: IMG projects SMPL-based motion vectors into latent guidance injected through learnable projections in intermediate DiT blocks.This multi-depth conditioning reinforces 3D awareness, anatomical plausibility, and viewpoint consistency.
Hierarchical conditioning: RASA combines continuous 2D calibration with deep-stage 3D guidance to decouple motion, geometry, and identity.The framework is designed for robust animation under severe structural discrepancies.
3.5 Cross-Identity Misalignment Benchmark: CIM-Bench pairs real human subjects and stylized synthetic characters performing identical motion patterns to test realistic cross-identity discrepancies.Its construction uses generated identities, collected portraits, synthesized videos, and rigorous evaluation motivation.

4 Experiments

RASA is evaluated on standard and cross-identity benchmarks using quantitative, qualitative, ablation, and efficiency analyses. Results show that SPC and IMG complement each other to improve structural alignment, motion consistency, and identity preservation.

Evaluation Settings: RASA is evaluated on TikTok and CIM-Bench using image-level and video-level metrics, including PSNR, SSIM, LPIPS, FID, FVD, and FID-VID.CIM-Bench testing excludes its test subset from training.
Comparison with State-of-the-Art Methods: On TikTok, RASA achieves the best image-level performance, while remaining competitive with larger or more heavily conditioned video-generation methods.It surpasses One-to-All-14B and Human-DiT on multiple metrics despite using a 1.3B backbone and fewer conditioning signals.
Comparison with State-of-the-Art Methods: On CIM-Bench, RASA consistently outperforms competing approaches across image-level and video-level metrics while preserving facial identity consistency.The benchmark introduces substantial pose–reference misalignment.
Ablation Study: Adding FCA to SMS improves PSNR from 18.90 to 20.86 on TikTok and reduces FID from 23.04 to 22.01.SPC also resolves spatial dislocations and preserves the reference character’s skeletal proportions.
Ablation Study: Adding IMG to SPC reduces TikTok FVD from 289.16 to 190.82 and improves FID-VID from 6.56 to 3.32.SPC supplies 2D spatial alignment, while IMG supplies anatomically grounded 3D guidance.
Efficiency Analysis: SPC and IMG add 19.78M parameters and only 2 seconds of latency, increasing generation time from 59s to 61s.The modules use 90.29G and 0.17G FLOPs, respectively, for a 41-frame 832 × 480 video on one H100 GPU.
Ablation Study: Using the driving latent as the FCA query preserves motion transfer, whereas using the reference latent as query weakens dynamic motion.The driving latent retrieves and broadcasts aligned identity features while retaining complex motion patterns.

5 Conclusion

RASA addresses cross-identity character animation by separating geometric calibration from semantic motion preservation. Its SPC and IMG modules, together with CIM-Bench, support robust evaluation under structural variation.

5 Conclusion: RASA addresses pose–reference misalignment through continuous geometric calibration and shape-agnostic, depth-aware motion guidance.SPC mitigates scale and proportion mismatches, while IMG injects anatomically grounded articulation cues into the diffusion backbone.
5 Conclusion: CIM-Bench provides a high-fidelity benchmark designed to evaluate realistic cross-identity structural variations.The benchmark complements evaluations on standard datasets.
5 Conclusion: Experiments on standard datasets and CIM-Bench show that RASA achieves state-of-the-art visual fidelity and motion stability.The conclusion presents disentangled spatial and motional priors as the framework’s organizing paradigm.

6 Limitation and Future Work

RASA handles extreme humanoid proportions but remains limited by its 1:1 body-part correspondence assumption and incomplete modeling of non-humanoid topologies. Hand and facial motion quality is also constrained because SMPL does not explicitly model those articulations.

Limitations: RASA successfully handles characters with extreme humanoid proportions, but Fig. 7 identifies failures for non-humanoid topologies.The figure marks successful extreme-proportion cases in green and non-humanoid failures in red.
Limitations: The framework assumes 1:1 body-part correspondence, limiting its suitability for multi-headed, multi-armed, or limbless characters.These topologies prevent reliable spatial correspondences from being established.
Future Work: Hand and facial motion quality is constrained because the framework relies on pose representations while SMPL lacks explicit facial-expression and hand-articulation modeling.The authors propose SMPL-X and MANO as future directions for explicit modeling.