PaperlayerView exact revision on arXiv

Source-linked AI summary

One Model to Rig Them All: Diverse Skeleton Rigging with UniRig

Jia-Peng Zhang, Cheng-Feng Pu, Meng-Hao Guo, Yan-Pei Cao, Shi-Min Hu

arXiv:2504.12451v1cs.GR

TL;DR

The growing volume and complexity of 3D models creates demand for efficient rigging, while existing methods can struggle with diverse topologies. UniRig predicts skeletons autoregressively and skinning weights with Bone-Point Cross Attention, using Rig-XL as a large, diverse dataset. The paper reports state-of-the-art results in skeleton and skinning-weight prediction across a wide range of object categories and skeletal structures.

  • Problem

    Automatic rigging must handle diverse 3D models, but existing approaches can be topology-limited, unstable, or reliant on manual intervention.

    Source: §1 · p. 2, §2.1.2 · p. 3

  • Method

    UniRig combines autoregressive skeleton prediction with Bone-Point Cross Attention for skinning weights and trains using the Rig-XL dataset.

    Source: §1 · p. 2

  • Results

    UniRig achieves state-of-the-art results in skeleton and skinning-weight prediction across a wide range of object categories and skeletal structures.

    Source: §1 · p. 2

  • Takeaways & Limitations

    UniRig is presented as a tool for researchers and practitioners, supporting diverse object categories, human-in-the-loop editing, and realistic animation.

    Source: §9 · p. 15

    UniRig’s performance is tied to the quality and diversity of its training data, and Rig-XL may not encompass the full range of possible skeletons.

    Source: §9 · p. 15

Abstract

from arXiv · show

The rapid evolution of 3D content creation, encompassing both AI-powered methods and traditional workflows, is driving an unprecedented demand for automated rigging solutions that can keep pace with the increasing complexity and diversity of 3D models. We introduce UniRig, a novel, unified framework for automatic skeletal rigging that leverages the power of large autoregressive models and a bone-point cross-attention mechanism to generate both high-quality skeletons and skinning weights. Unlike previous methods that struggle with complex or non-standard topologies, UniRig accurately predicts topologically valid skeleton structures thanks to a new Skeleton Tree Tokenization method that efficiently encodes hierarchical relationships within the skeleton. To train and evaluate UniRig, we present Rig-XL, a new large-scale dataset of over 14,000 rigged 3D models spanning a wide range of categories. UniRig significantly outperforms state-of-the-art academic and commercial methods, achieving a 215% improvement in rigging accuracy and a 194% improvement in motion accuracy on challenging datasets. Our method works seamlessly across diverse object categories, from detailed anime characters to complex organic and inorganic structures, demonstrating its versatility and robustness. By automating the tedious and time-consuming rigging process, UniRig has the potential to speed up animation pipelines with unprecedented ease and efficiency. Project Page: https://zjp-shadow.github.io/works/UniRig/

1 INTRODUCTION

The growth of AI-generated 3D content increases demand for efficient rigging, while existing approaches face limits in topology flexibility, stability, or compatibility. UniRig combines autoregressive skeleton prediction, Skeleton Tree Tokenization, and bone-point cross-attention, and is evaluated using the diverse Rig-XL dataset.

1 INTRODUCTION

  • Manual rigging remains a time-consuming, expertise-intensive bottleneck as AI-driven 3D content creation generates complex models at increasing scale.The introduction frames automatic rigging as a response to demand across AI-powered and traditional workflows.Source: §1 · p. 2
  • Template-based methods depend on predefined skeleton topologies, while template-free methods can produce unstable or topologically implausible skeletons.Template-free skeletons can also make motion retargeting challenging.Source: §1 · p. 2
  • Rigging methods still struggle with diverse models because they may be category-limited, challenged by complex topologies, or require manual intervention.UniRig is proposed as a learning-based framework for rigging diverse 3D models.Source: §1 · p. 2
  • Skeleton Tree Tokenization encodes hierarchical parent-child relations as tokens, guiding autoregressive generation toward topologically valid skeletons.The tokenization also represents bone types, including spring bones and template bones, for downstream motion retargeting.Source: §1 · p. 2
  • Rig-XL contains over 14,000 3D models with diverse skeletal structures and corresponding skinning weights; VRoid is used to refine handling of detailed characters.The paper presents Rig-XL as a large-scale dataset and also uses the anime-character dataset VRoid.Source: §1 · p. 2
  • UniRig combines autoregressive skeleton prediction with Bone-Point Cross Attention for skinning weights, and achieves state-of-the-art results across object categories and skeletal structures.The contribution statement describes performance in both skeleton prediction and skinning weight prediction.Source: §1 · p. 2

2 RELATED WORKS

Related work spans data-driven mesh deformation and automatic rigging, with different trade-offs in skeleton representation, workflow compatibility, and topology handling. Existing approaches range from geometric methods requiring animator input to learned methods that can struggle with category limits or bone connectivity.

2.1 Data-Driven Mesh Deformation Transfer

  • Data-driven mesh deformation transfer has emerged alongside skeletal animation, with mesh-deformation methods bypassing traditional rigging processes.The related-work overview frames these methods within character animation research.Source: §2.1 · p. 2

2.1.1 Skeleton-free Mesh Deformation.

  • Skeleton-free mesh-deformation methods include pose transfer, iterative coarse-to-fine prediction, motion generation, neural pose transfer, cycle-consistent deformation, and zero-shot retargeting.SfPT, HMC [Wang et al. 2023a], Tapmo [Zhang et al. 2023a], [Wang et al. 2020],, and ZPT [Wang et al. 2023b] exemplify these approaches.Source: §2.1.1 · p. 3, §2.1.2 · p. 3

2.1.2 Vertex Displacement Prediction.

  • Skeleton-free and direct vertex-displacement approaches face integration challenges in workflows built around traditional skeletal rigging and animation.Source: §2.1.2 · p. 3

2.2 Automatic Rigging Methods

  • Automatic rigging methods include traditional geometry-based approaches and deep-learning techniques.Both aim to create a skeleton and associate it with a 3D mesh.Source: §2.2 · p. 3

2.2.1 Traditional Geometric Methods.

  • Traditional geometric methods can handle complex topologies but may require substantial animator intervention before production.LazyBones, based on medial-axis fitting, still requires considerable animator input to fine-tune skeletons.Source: §2.2.1 · p. 3

2.2.2 Deep Learning Algorithms.

  • Deep-learning rigging methods predict joints through heatmaps, motion features, residual deformation, or diffusion, but often infer bone connectivity separately, risking topological errors.RigNet [Xu et al. 2020], MoRig, Neural Blend Shapes, and DRiVE [Sun et al. 2024] illustrate these approaches; methods may also be topology-specific or trade flexibility against stability and precision.Source: §2.2.2 · p. 3

3 OVERVIEW

UniRig is a two-stage framework for rigging diverse 3D models, using autoregressive skeleton-tree prediction followed by skin-weight prediction conditioned on the skeleton. Its evaluation compares it with state-of-the-art methods and commercial tools, and the paper presents practical applications in assisted rigging and animation.

3 OVERVIEW

  • UniRig predicts a skeleton tree autoregressively from an input mesh, then predicts per-point skin weights conditioned on that skeleton.The second stage uses Bone-Point Cross Attention.Source: §3 · pp. 3–4
  • Rig-XL examples illustrate well-defined skeleton structures.Source: Figure 2 · p. 4
  • VRoid supports refinement for fine-detail anime-style models, while Rig-XL spans over 14,000 diverse, high-quality models to support generalization across categories.Source: §3 · p. 4
  • The evaluation compares UniRig with state-of-the-art methods and commercial tools, and the paper showcases human-assisted rigging and character animation applications.Source: §3 · p. 4

4 DATASET

The section describes the curation of VRoid and Rig-XL and the preprocessing used to assemble a diverse dataset of rigged 3D models.

4.1 VRoid Dataset Curation

  • VRoid comprises 2,061 anime-style 3D models curated for detailed, expressive skeletal rigs, particularly for human-like characters.The dataset complements Rig-XL with models featuring detailed skeletal structures.Source: §4.1 · p. 4

4.1.1 VRM Format.

  • VRM standardizes humanoid skeletons compatibly with Mixamo and supports spring bones for simulated motion in hair, clothing, and tails.Spring bones let the model learn rigs that support dynamic effects.Source: §4.1.1 · p. 4

4.2 Rig-XL Dataset Curation

  • Rig-XL derives from Objaverse-XL, whose static models lack the consistent skeletons and skinning weights needed for rigging training.The authors initially selected 54,000 movable, higher-quality models, but found scene animations, missing rig data, and human-model bias among them.Source: §4.2 · p. 4

4.2.1 Dataset Preprocessing.

  • Preprocessing filtered for connected skeletons with 10–256 bones, categorized and deduplicated models, then manually checked skeletal structures and data.Categorization used rendered-model captions from ChatGPT-4o; manual review corrected errors such as bone edges incorrectly marked as disconnected.Source: §4.2.1 · p. 5

4.2.2 Dataset Details.

  • 14,611 unique 3D models with well-defined skeletons and complete skinning weights make up Rig-XL after preprocessing.Human-related models remain dominant; skeleton counts have modes at 52 and 28, corresponding to Mixamo models with and without hands.Source: XL dataset · p. 5

5 AUTOREGRESSIVE SKELETON TREE GENERATION

UniRig generates skeleton trees autoregressively, conditioning joint predictions on previously generated joints and mesh geometry. Its tokenization encodes coordinates and hierarchy compactly, with specialized representations for known bone types and general branches.

5 AUTOREGRESSIVE SKELETON TREE GENERATION

  • UniRig predicts skeleton trees sequentially, conditioning each joint on prior predictions and geometric features from the input mesh.The output includes joint positions and parent relationships that define tree connectivity.Source: §5 · p. 5

5.1 Skeleton Tree Tokenization

  • The tokenization discretizes bone coordinates into 256 values and uses dedicated identifiers to encode bone-sequence types.For template bones, template identifiers allow parent coordinates to be inferred from the template definition.Source: §5.1 · p. 7
  • Known templates and spring-bone chains can be encoded as compact sequences, avoiding repeated coordinate information and representing bone types explicitly.Type identifiers provide information intended to help the model learn and generalize across skeletal structures.Source: §5.1 · p. 7
  • For bones without an identifiable type, DFS extracts linear chains as subsequences, marks branches, and sorts children by tail coordinates for consistent ordering.During decoding, chains are connected by merging joints whose coordinates fall within a predefined distance threshold.Source: §5.1 · p. 7
  • 27.47% on VRoid and 29.72% on Rig-XL are the average token reductions achieved by optimized tokenization.The reduced representation also lowers training memory use and speeds inference.Source: Table 2 · p. 8, §5.1 · p. 8

6 SKIN WEIGHT PREDICTION VIA BONE-POINT CROSS ATTENTION

UniRig predicts skinning weights and bone attributes through bone–point cross-attention, addressing the scale of meshes with many vertices and bones. Its training balances bone contributions and uses simulated motion to encourage realistic deformation.

6 SKIN WEIGHT PREDICTION VIA BONE-POINT CROSS ATTENTION

  • Bone–point cross-attention models relationships between the predicted skeleton and input mesh to predict skinning weights and bone attributes.Bone and point encoders produce features; point features query bone features for weight prediction, with voxel geodesic distance added before decoding and softmax normalization.Source: §6 · p. 8

6.1 Training Strategy Based on Skeletal Equivalence

  • Uniform surface sampling can favor bones in densely sampled regions, while skeletal-equivalence training aims to give each bone equal influence.It randomly freezes a subset of bones and normalizes each bone’s loss by the vertices it influences, preventing bones affecting many vertices from dominating.Source: §6.1 · p. 9

6.2 Indirect Supervision via Physical Simulation

  • Spring bones produce more natural hair and skirt movement than rigid-body motion alone.Figure 6 illustrates the animation difference with and without spring bones.Source: Figure 6 · p. 9, §6.2 · p. 9
  • Indirect supervision compares simulated motion from predicted and ground-truth weights and bone attributes to guide learning toward more realistic deformation.The method uses differentiable Verlet simulation inspired by VRoid spring-bone dynamics; it assumes uniform downward gravity, neglects collisions, and uses a short sequence of T=3.Source: §6.2 · pp. 9–10

7 EXPERIMENTS

UniRig is evaluated on bone prediction, skinning weights, and mesh deformation, with comparisons to academic methods and qualitative comparisons to commercial tools. Ablations examine its tokenization and training strategies.

7.1 Implementation Details

  • Training balances category sampling to reduce Rig-XL’s overrepresentation of human-related models, and uses rotation, scaling, and motion-based augmentation.The two-stage training process predicts skeleton trees and then skinning weights; the latter uses a frozen pretrained point transformer with a reduced trainable model.Source: §7.1.1 · p. 10, §7.1.2 · p. 10

7.2 Results and Comparison

  • Evaluation uses VRoid and Rig-XL validation samples for skeleton prediction, while commercial-tool comparisons are qualitative because those systems are closed-source.The validation set contains 50 VRoid and 100 Rig-XL samples; models are compared visually for skeleton quality and resulting animation against commercial tools.Source: §7.2 · p. 10, §7.2.1 · p. 10
  • UniRig outperforms other methods across datasets on J2J joint-position accuracy, per-vertex L1 skinning-weight error, and mesh-deformation reconstruction loss.The reported comparisons appear in Tables 3, 4, and 5, respectively.Source: §7.2.1 · p. 11, Table 3 · p. 11, Table 4 · p. 11

7.2.1 Bone Prediction.

  • Qualitative comparisons show more detailed and accurate VRoid skeletons and more realistic motion, including hair and hand movement, than the compared alternatives.Comparisons include RigNet, NBS, and TA-Rig for skeletons, and commercial tools and NBS for motion; NBS was fine-tuned on VRoid and Accurig required manual correction.Source: §7.2.1 · p. 11, Figure 7 · p. 11, Figure 9 · p. 12

7.3.1 Tokenize Strategy.

  • The tokenization ablation reports that the optimized strategy shortens token sequences, reduces inference time, and improves bone-prediction accuracy across datasets.Table 6 reports the comparison for models trained for 160 epochs, with inference time tested on one RTX 4090 GPU.Source: §7.3.1 · p. 13, Table 6 · p. 13

7.3.2 Indirect Supervision based on Physical Simulation.

  • On VRoid, indirect supervision improves both deformation error and skinning-weight error compared with training without physical simulation.The ablation focuses on VRoid because its spring bones are directly affected by the simulation.Source: §7.3.2 · p. 13, Table 7 · p. 14
  • The full skeletal-equivalence strategy achieves the best reconstruction loss; removing either bone freezing or bone-loss normalization degrades performance.The ablation tests both components individually.Source: §7.3.3 · p. 14, Table 8 · p. 14

8 APPLICATIONS

UniRig supports user editing of predicted skeleton trees followed by regeneration of the rig. It also produces spring-bone motion for VRM-compatible characters, including use in virtual streaming.

8.1 Human-Assisted Auto-rigging

  • Users can edit a predicted skeleton tree and regenerate the affected rig, for example by adding a tail branch or removing unwanted spring bones.The iterative workflow combines automatic prediction with user customization and correction.Source: §8.1 · p. 14, Figure 11 · p. 14

8.2 Character Animation

  • UniRig predicts spring-bone parameters and generates VRM-compatible models from meshes for animation platforms, including VTubing use.The paper demonstrates a UniRig-rigged character with smooth spring-bone motion in Warudo.Source: §8.2 · p. 14, Figure 12 · p. 14

9 CONCLUSIONS

UniRig combines skeleton prediction and skinning-weight prediction in a learning-based rigging framework, trained using Rig-XL and curated VRoid data. Its generalization remains bounded by the quality and diversity of its training data.

9 CONCLUSIONS

  • UniRig combines a novel tokenization strategy and two-stage training to achieve state-of-the-art skeleton and skinning-weight prediction results.Rig-XL and curated VRoid data support training across varied object categories and skeletal structures.Source: §9 · p. 15
  • Performance may be suboptimal for objects that substantially differ from training data, including highly unusual skeletal structures and abstract or highly stylized characters.The authors identify the quality and diversity of training data as a constraint and suggest user edits and expanded data as ways to refine generalization.Source: §9 · p. 15
  • The paper concludes that diverse-category rigging, human-in-the-loop editing, and realistic animation make UniRig useful to researchers and practitioners.Source: §9 · p. 15

A.1.1 Rig-XL Data Process.

The appendix describes Rig-XL data cleanup, VRM spring-bone simulation, and further qualitative and quantitative comparisons of UniRig.

A.1.1 Rig-XL Data Process.

  • The preprocessing repairs unreasonable root topology by making the heavy child the new root and connecting the remaining forest with a Euclidean-distance minimum spanning tree.The described filtering rule flags trees whose root out-degree exceeds 4 and whose heavy-child subtree exceeds half the tree size.Source: §A.1.1 · p. 17

A.2.1 Capture outlier through reconstruction loss.

  • Training removes reconstruction-loss outliers above 10 times the current average and samples with points whose skinning weights are entirely missing.The authors say softmax applied per point cannot fit cases where all weights are zero.Source: §A.2.1 · p. 17

A.3 Methods

  • VRM body motion begins with forward kinematics using the standard Mixamo template, followed by sequential top-to-bottom Verlet integration for spring bones.The sequential spring-bone calculation is described as producing coherent animation.Source: §A.3.1 · p. 17

A.3.1 Physical Simulation on VRM.

  • Spring-bone positions are computed in chain order, from top to bottom, after the body's basic motion is calculated.The appendix identifies this procedure as the physical simulation used when deforming the VRM body.Source: §A.3.1 · p. 17

A.4 More Results

  • UniRig predicts detailed skeletons and skinning weights across varied examples, with reported gains over NBS and RigNet in skinning weights and complex cases such as ants.The appendix also presents comparisons with NBS and RigNet for skeletons and skinning weights, plus quantitative tables on skeleton prediction and Chamfer distances.Source: §A.3.1 · p. 17, Figure 13 · p. 17, Figure 14 · p. 17
Loading 2504.12451v1…