Source-linked AI summary
Inverse Rig Optimization from Line Drawings
Zihao Zhu, Yuki Koyama
TL;DR
Stylized 3D animation is laborious because artists must search through non-intuitive rig controls to match screen-space contours. The paper recovers editable rig parameters from redrawn contour strokes using a differentiable MLP surrogate and screen-space optimization. Across diverse characters and scenarios, it reproduces artist strokes within seconds per keyframe, while depth ambiguity limits recovery for larger deformations.
Problem
Stylized animation is largely hand-authored through repeated rig-control adjustments because the desired contour is not directly exposed as an input.
Method
The method matches user-drawn contours to projected mesh contours and backpropagates screen-space loss through a pre-trained differentiable NeuralRig surrogate to update high-level rig parameters.
Results
Across cartoon-style and anime-style characters, including facial expressions, props, view-dependent deformation, and perspective exaggeration, the method reproduces artist strokes within seconds per keyframe.
Takeaways & Limitations
The framework provides a line-driven way to refine shot-specific stylization while preserving editable production-rig controls.
Takeaways & Limitations
Because 2D strokes lack depth information, multiple rig configurations can match the same contour and produce an unintended pose when larger deformations are possible.
Abstract
from arXiv · showhide
Stylized 3D character animation is largely hand-authored, with animators authoring rig parameters one keyframe at a time to find the best pose. Because stylized work reads chiefly through contour lines, drawing contours in the camera view is the most direct and precise way to express artistic intent. This mismatch between the rig controls and the artist's goal forces a laborious trial-and-error workflow, with animators repeatedly manipulating rig controls against the rendered view to match the desired contour. To address this, we propose a method that recovers rig parameters from screen-space contour strokes, enabling effective keyframing from sketches. Given strokes that redraw the current contour, our method optimizes the high-level rig parameters defined in the DCC tool. The key is to use a pre-trained MLP rig surrogate that provides a differentiable map from rig parameters to mesh vertices, replacing the original black-box rig within the optimization process. We match user-drawn lines to mesh contour lines and backpropagate the resulting screen-space error through the surrogate to update the rig parameters. Our results demonstrate that the method works for diverse characters and practical scenarios.
1. Introduction
The paper targets the trial-and-error process of posing stylized characters by treating artist-redrawn screen-space contours as direct control input. A differentiable rig surrogate recovers editable high-level rig parameters from those strokes, and the method works across diverse practical scenarios.
- Motivation: Stylized animation requires manually authoring each keyframe because non-physical effects cannot be captured or simulated, while numerical rig controls provide little guidance about the resulting pose.Animators repeatedly adjust controls, render, compare against the intended look, and repeat across many parameters.
- Motivation: Contours provide an economical and sufficient signal for stylized appearance because the style is rooted in line-based 2D animation and toon shading minimizes the importance of interior surface detail.The paper therefore treats artist-redrawn contours as the control input for recovering rig-parameter updates.
- Approach: The method recovers interpretable, re-editable high-level rig controller parameters rather than producing a baked mesh.These parameters are the controls animators use in DCC tools, while the original rig continues to drive the final production deformation.
- Approach: A pre-trained NeuralRig surrogate maps rig parameters to object-space vertices, enabling contour matching and screen-space loss optimization through backpropagation.The surrogate supplies gradients during optimization; recovered parameters are written back into the DCC to drive the original rig.
- Results: The method recovers rig parameters that reproduce artist strokes within seconds per keyframe across cartoon-style and anime-style characters and several practical scenarios.The reported settings include facial expressions, rigged props, view-dependent deformation, and perspective exaggeration.
- Contributions: The contributions include direct inversion of production controls, line-driven posing from rendered-view strokes, and a formulation applicable regardless of the rig’s internal deformation mechanism.The output remains interpretable and editable through the standard rig interface.
2. Related Work
Prior work addresses rig inversion, neural rig surrogates, sketch-based control, and drawing-based pose reconstruction. This paper’s related-work context spans both generic parametric models and methods operating on rigged characters or production-oriented deformations.
- Rigging foundations: Rig deformation commonly uses blendshapes and linear blend skinning, while other deformers can also serve as rig building blocks.Blendshapes combine sculpted shape targets, whereas linear blend skinning blends skeletal joint transforms per vertex.
- Parametric models: Generic parametric models such as 3DMM, FLAME, and SMPL simplify animation research but are expressively more limited than production rigs for stylized animation.These models summarize broad shape categories using relatively few parameters.
- Rig inversion: Prior rig-inversion methods infer blendshape sliders, facial deformations, joint configurations, or black-box rig parameters from semantic controls, motion markers, or sketches.The cited approaches cover direct blendshape manipulation, learned facial correctives, regression to rig parameters, and sketch-based rig fitting.
- Neural surrogates: Neural forward surrogates have been developed to approximate expensive film-quality body, facial, skeletal, and corrective deformations, primarily to accelerate forward evaluation or improve portability and fidelity.The paper positions differentiability as an additional use of such surrogates for inversion.
- Sketch-based control: Sketch-based character control includes mapping cursive sketches to motion, reconstructing articulated poses, fitting meshes to occlusion contours, and controlling rigged sketch abstractions.These works establish drawing as an interface for 3D character manipulation.
3. Method
The method recovers editable high-level rig parameters from redrawn screen-space contours through a differentiable surrogate and hierarchical contour matching. It restricts optimization to relevant controls and supports flexible contour-path inputs for efficient inversion.
- 3.1. Overview: The DCC supplies the current rig parameters, camera matrices, and fixed contour edge paths, while the animator redraws those contours as screen-space strokes.The recovered parameter vector is written back into the DCC as the new pose.
- 3.1. Overview: The framework permits any informative vertex sequence for E, including detected contours, selected edges, artist-selected screen-space paths, or pre-authored polylines.In the Blender implementation, users can select relevant detected contours and keep E fixed throughout optimization.
- 3.1. Overview: Contour Rendering projects selected contour vertices through the model, view, and projection matrices, and matching pairs the resulting polylines with the animator’s strokes.The screen-space loss is backpropagated through projection and the NeuralRig while Adam updates the rig parameters.
- 3.2. The Neural Rig: A pre-trained NeuralRig MLP replaces the non-differentiable production rig by mapping rig parameters to object-space vertex positions.It is trained per rigged mesh to mimic the production rig’s forward map.
- 3.3. Rig Influence Map: A precomputed influence map forms an active control set, allowing gradient updates only for rig controls that move vertices on the selected contour paths.The active set is computed once because the influence map and contour paths remain fixed.
- 3.4. Matching: Hierarchical matching first pairs strokes with contour polylines, then establishes ordered point-to-vertex correspondences within each pair.Resampling onto a shared arc-length grid and orientation alignment support the line-level shape comparison.
4. Results
The method recovers stylized poses from contour strokes across facial animation, object posing, view-specific deformation, and perspective exaggeration. These examples show editable rig outputs and coordinated or camera-specific changes driven by drawings.
- Facial Expression Keyframing: Each recovered face keyframe starts from the rest pose and uses only the artist’s strokes as per-pose input.Across two cartoon-style face rigs and three expressions, projected contours land on the user’s strokes while localized edits affect brows and mouths.
- Facial Expression Keyframing: The recovered facial keyframes remain editable through the character’s standard rig interface rather than becoming baked meshes.The output is a rig-parameter vector that permits further animator refinement.
- Contour-Defined Object Posing: A single set of strokes drives coordinated adjustments across many rig controls for contour-defined object poses.The skirt example demonstrates a pose that would otherwise require searching through hundreds of near-redundant degrees of freedom.
- View-Specific Shape Stylization: The method recovers view-specific face-contour corrections from a single stroke drawn in the target camera view.The anime face example changes the projected jaw-cheek contour to restore the intended camera-dependent curvature.
- Stylized Perspective: Perspective exaggeration is achieved by redrawing the desired contour, producing geometry that reads correctly from the drawing camera despite distortion elsewhere.In the katana example, the recovered geometry is intentionally distorted and stretches beyond natural length from an external view.
5. Discussion
The method has several scope boundaries: depth ambiguity, fixed contour paths, surrogate accuracy, single-keyframe optimization, static deformation, and multi-LOD production integration.
- 5.1. Limitations: 2D strokes lack depth information, so multiple rig configurations can match one contour when the rig permits substantial deformation.The recovered pose may differ from the artist’s intent even when the screen-space loss decreases.
- 5.1. Limitations: Fixed contour-path identities suit local shot-level edits but can fail when larger deformations change visibility or contour topology.A selected path may align with the stroke although a different edge set forms the true silhouette.
- 5.1. Limitations: Surrogate accuracy limits recovered-pose accuracy, especially for poses far from the NeuralRig training distribution.Errors in the learned approximation can make the production rig reproduce strokes less faithfully than optimization predicted.
- 5.2. Future Work: The current pipeline solves one keyframe at a time; jointly optimizing sequences with temporal smoothness is proposed as future work.A smoothness loss on adjacent rig parameters could reduce frame-to-frame jumps.
- 5.2. Future Work: Static NeuralRig mappings cannot represent trajectory-dependent secondary motion such as hair, cloth, or soft-body dynamics.The method therefore does not apply when deformation is delegated to an in-loop simulator.
- 5.2. Future Work: Production-ready multi-LOD integration remains future work because each level of detail would require a separate NeuralRig.The paper also leaves studio-pipeline evaluation open.
6. Conclusion
The paper presents a framework for shot-level stylization from artist-drawn contour strokes. It propagates contour edits through a differentiable rig surrogate to recover rig controls across several stylized-character applications.
- 6. Conclusion: The framework recovers rig parameters from redrawn contours using a differentiable surrogate, stroke-to-contour matching, and a screen-space loss.The recovered controls refine shot-specific character appearance through artist-drawn lines.
Appendix A: NeuralRig Accuracy
The appendix evaluates the three-dimensional accuracy of a facial NeuralRig using visualized per-vertex errors and their distribution over held-out poses.
- Appendix A: NeuralRig Accuracy: The facial NeuralRig’s per-vertex three-dimensional error is visualized as a surface heatmap.Figure 8 provides a spatial view of prediction error across the mesh.
- Appendix A: NeuralRig Accuracy: The corresponding per-vertex three-dimensional error distribution is evaluated over 10,000 held-out poses.Figure 9 reports the cumulative distribution function for these errors.
Appendix B: Additional Experimental Details
The appendix documents experimental details and additional results for four applications: facial-expression keyframing, contour-defined object posing, view-specific shape stylization, and stylized perspective.
- Appendix B: Additional Experimental Details: Figure 10 presents representative results for the four applications introduced in Section 4.The applications span facial expressions, object posing, view-dependent shape stylization, and perspective exaggeration.
- Appendix B: Additional Experimental Details: Table 1 summarizes model configurations, training settings, input strokes, matched contour vertices, optimization steps, and total optimization time.The table organizes these details for the four experiments shown in Figure 10.
Appendix C: Extreme Input Strokes
Moderate differences between input strokes and existing contours can remain tractable, but extreme cases may produce implausible 3D configurations despite aligned projected contours.
- Ambiguous stroke-to-contour assignments can be specified by the user.
- Extreme input strokes can fall outside the method’s primary operating regime of local, shot-level refinement.
- Mesh topology, rig expressiveness, depth ambiguity, and optimization limitations can yield implausible 3D configurations even when projected contours align.
Appendix D: Freehand Input: Raw and Preprocessed Strokes
The main experiments use cleaned strokes, while raw freehand strokes produce a visually imperceptible difference in the tested comparison.
- The main-paper experiments use preprocessed strokes with approximately uniform spacing and reduced high-frequency noise.
- The intended users are experienced 2D artists whose corrective strokes are expected to be relatively clean.
- Raw freehand strokes produce a visually imperceptible difference from preprocessed strokes in the tested example.
- Matching is primarily sensitive to overall stroke trajectory because target contours contain far fewer vertices than input strokes.
- Small fluctuations have limited effect unless they create false corner detections during point-level matching.