PaperlayerView exact revision on arXiv

Source-linked AI summary

AnimalLift: Reconstructing Animatable 3D Animals from a Single Image by Learning Canonical Shape, Texture, and Fur Maps

Chunyi Sun, Ruyi Zha, Weijian Deng, Junlin Han, Dylan Campbell, Stephen Gould

arXiv:2609.09513v1cs.CV

TL;DR

Reconstructing an animation-ready animal from one image requires structured geometry, editable appearance, and fur compatible with deformation. AnimalLift predicts these elements in a shared canonical UV space and demonstrates strong reconstruction quality and cross-category generalization, while supporting downstream animation and fur workflows.

  • Problem

    Single-image animal reconstruction methods often lack the shared topology, editable appearance, and explicit fur structure needed for animation-ready assets.

  • Method

    AnimalLift lifts an input image into a shared canonical space and jointly predicts canonical geometry, UV texture, and UV-aligned fur using procedural aligned supervision.

  • Results

    Experiments on synthetic and real-world datasets demonstrate strong reconstruction quality and generalization across animal categories.

  • Takeaways & Limitations

    The structured representation supports downstream animation, pose transfer, fur editing, and simulation-compatible rendering.

  • Takeaways & Limitations

    The method assumes shared topology, is limited to a family of quadrupeds, and may miss fine identity cues such as scars or broken strands.

Abstract

from arXiv · show

Reconstructing a fully animatable 3D animal from a single image remains challenging because animation-ready assets require not only plausible geometry, but also a unified topology, editable appearance, and fur representations compatible with deformation and simulation. Existing image-to-3D approaches often rely on implicit or loosely structured representations that are difficult to rig or edit, while parametric animal models support animation but cannot capture detailed texture and fur appearance. We present AnimalLift, a framework for reconstructing structured, animation-compatible 3D animal assets with explicit fur from a single image. Our method lifts an input image into a shared canonical space with a consistent topology and UV parameterization across the dataset, enabling joint prediction of canonical geometry, texture, and fur in a unified feed-forward architecture. A key component of our representation is a UV-aligned fur map that encodes strand geometry in a surface-aligned canonical domain, allowing explicit fur reconstruction compatible with mesh deformation and fur simulation. To train the model, we introduce a procedural data generation pipeline that provides large-scale supervision with aligned geometry, texture, and fur across diverse animal species and appearances. Experiments on synthetic and real-world datasets demonstrate strong reconstruction quality and generalization across animal categories. Beyond reconstruction, our structured representation directly supports downstream applications including animation, pose transfer, fur editing, and simulation-compatible rendering.

1 Introduction

AnimalLift addresses the difficulty of reconstructing animation-ready animals by combining canonical structure with detailed geometry, texture, and explicit fur from a single image. Its unified representation and procedural supervision support reconstruction quality, generalization, and downstream graphics applications.

  • 1 Introduction: Shared topology and canonical correspondences make outputs easier to rig, animate, and reuse than unconstrained or implicit reconstructions.Consistent vertex, body-region, and UV meanings also allow supervision to be aggregated across instances.
  • 1 Introduction: A UV-aligned fur map represents strand geometry as a surface-aligned structure compatible with mesh deformation and fur simulation.This avoids treating fur solely as a view-dependent rendering effect.
  • 1 Introduction: AnimalLift reconstructs canonical mesh geometry, texture, and explicit fur jointly in a shared UV-aligned space from one image.The representation preserves consistent topology and parameterization while separating appearance from pose.
  • 1 Introduction: The procedural data pipeline supplies aligned geometry, texture, and fur supervision across diverse animal species, shapes, and appearances.It maintains shared topology and UV correspondence while synthesizing varied assets.
  • 1 Introduction: Experiments on synthetic and real-world datasets show strong reconstruction quality and generalization across animal categories.The structured representation also supports animation, pose transfer, fur editing, and simulation-compatible rendering.

2 Related Work

Related work spans general image-to-3D reconstruction, structured animal models, automatic rigging, and fur modeling. AnimalLift is positioned against methods that lack shared canonical structure, detailed appearance, or direct animation-compatible fur representations.

  • Image-to-3D Reconstruction: Image-to-3D methods increasingly synthesize compelling assets, yet implicit fields, point representations, and unconstrained meshes lack canonical correspondences and editable parameterizations.These limitations make direct rigging, animation, and reuse in standard graphics pipelines difficult.
  • Parametric Animal Models: Parametric animal models provide shared topology and support deformation, but struggle with fine geometry, detailed textures, and high-frequency fur.They offer structure for animation while limiting appearance fidelity.
  • Automatic Rigging: Automatic rigging methods predict skeletons and skinning weights after reconstruction, whereas shared-topology assets can be rig-compatible by construction.The latter reduces dependence on mesh consistency and quality during post-processing.
  • Hair and Fur Modeling: Learning-based fur methods commonly predict hair from images, but animal-fur approaches often require multiview input and lack canonical-surface alignment for animation.This leaves a gap between strand reconstruction and deformation-compatible asset production.

3 Canonical Data Generation

AnimalLift generates aligned canonical training assets by combining shared-topology animal templates, UV-space texture and fur synthesis, and posed rendering with canonical supervision. The resulting dataset covers multiple quadruped categories and provides paired observations with structured targets.

  • Canonical Shape Space: The pipeline uses shared-topology quadruped templates with common skeleton correspondence and UV layouts across anatomical regions.Shape variation is added while preserving dense vertex correspondence and a common rig and UV domain.
  • Canonical Texture and Fur Generation: Textures are generated and varied directly in the canonical UV domain using geometry- and morphology-matched references.The external texturing systems are used only for offline data generation, not inference.
  • Canonical Texture and Fur Generation: Canonical fur is procedurally generated from shared grooming templates with density, length, curliness, and stochastic UV-space variation.Each strand is rooted on the canonical mesh and represented by root-relative offsets for resampled points.
  • Rendering and Pose Augmentation: Training observations are produced by articulating canonical assets, applying linear blend skinning, and randomizing viewpoints, camera distances, and HDR lighting.Pose, camera, and illumination affect only the rendered input; all targets remain canonical.
  • Canonical Supervision and Representation: Each sample pairs a rendered image with canonical mesh, UV texture, fur density, and UV-aligned fur tensors, and the dataset contains 11K samples across six animal categories.The fur tensor stores strand offsets while density identifies non-fur UV cells.

4 AnimalLift

AnimalLift predicts canonical mesh, texture, and UV-aligned fur from one image using a shared image encoder and specialized reconstruction heads. Its structured outputs support standard asset workflows because geometry, appearance, and fur remain aligned in canonical space.

  • Method Overview: AnimalLift reconstructs canonical mesh, UV texture, and UV-aligned fur from a single image through three specialized heads sharing a DINOv2 backbone.The geometry, fur, and texture heads operate on shared image features while producing outputs in canonical space.
  • Geometry Reconstruction: The geometry head predicts vertex offsets from a mean template, while attentive pooling produces a global descriptor for canonical mesh reconstruction.The mesh is recovered as the mean template plus predicted vertex offsets and trained with vertex, edge, Laplacian, and regularization terms.
  • Fur Reconstruction: The fur head preserves spatial structure by decoding dense UV maps containing root-relative strand offsets and per-cell strand density.The representation uses 63 resampled strand points per UV cell, with density indicating how many child strands to interpolate.
  • Texture Synthesis via Flow Matching: Texture synthesis uses flow matching in a compressed latent space to model multimodal high-frequency appearance rather than directly regressing UV colors.A convolutional autoencoder compresses textures, and the flow network synthesizes texture latents conditioned on image-derived appearance context before decoding.
  • Training: Two-stage training first fits the texture autoencoder, then jointly trains geometry, fur, and texture heads over a shared image-conditioned backbone.Joint training lets the heads share cues such as silhouette, species, proportions, and fur direction in one forward pass.
  • Production Integration: Shared topology, skeleton correspondence, and UVs allow reconstructed meshes to transfer to common rigs, while curve-based fur exports support native grooming, physics, and rendering.UV-defined strand roots remain anchored to the canonical surface, and Blender can interpolate full grooms from predicted guide strands without manual re-grooming.

5 Experiments

The experiments demonstrate reconstruction outputs across species and lighting, together with downstream animation, simulation, grooming, and editing applications. The reported assets combine shared-topology geometry, UV appearance, and explicit strand-based fur for production workflows.

  • Applications: The reconstructed assets support animation and pose transfer because their meshes share topology and UVs, enabling transfer to a common rig.The representation is intended for standard DCC workflows, including rigging and animation.
  • Reconstruction Results: AnimalLift produces single-image 3D animal reconstructions that preserve geometry, identity, and coherent hair patterns across species and natural lighting conditions.Figure 3 uses natural-scene renderings and zoom-ins to expose appearance and fur structure.
  • Applications: Figure 4 presents physical collision, wind simulation, and controllable hair and color editing using explicit mesh and strand-based hair reconstructed from one image.These examples illustrate applications enabled by the structured output rather than image-space rendering alone.
  • Applications: The UV-aligned fur field supports grooming, collision, gravity, and wind simulation while shared UV space enables aligned edits to color, fur length, and grooming style.The paper also reports transferring fur geometry and texture maps across animal classes with substantially different appearances.

5.3 Evaluation Metrics

AnimalLift is evaluated on synthetic and real-world test sets using metrics covering geometry accuracy, rendered appearance, and explicit fur quality. Geometry uses aligned canonical meshes, while appearance is assessed from multi-view renderings with perceptual and distributional measures.

  • Test Sets: Evaluation uses 640 real photographs and a held-out synthetic split with ground-truth meshes and fur annotations.The two test sets target real-world generalization and quantitative reconstruction assessment, respectively.
  • Evaluation Scope: The metrics cover three axes: geometry accuracy, rendered appearance, and explicit fur quality.This evaluation separates surface reconstruction from image-level appearance and strand-level fur reconstruction.
  • Geometry Metrics: Synthetic geometry is measured with Chamfer-L1 after similarity ICP alignment and unit-bounding-box normalization, with canonical outputs compared directly to canonical ground truth.The alignment accounts for differing scales and coordinate systems among methods.
  • Appearance Metrics: Rendered appearance is evaluated from six uniformly sampled views using NIQE, LPIPS, MV-CLIP, FIDCLIP, and Patch-FID.The metrics assess naturalness, perceptual similarity, semantic consistency, and distribution-level appearance quality.

5.4 Reconstruction Quality Comparison

AnimalLift achieves competitive reconstruction quality while producing structured canonical assets with explicit fur, supporting animation and downstream editing or simulation workflows.

  • AnimalLift achieves competitive reconstruction quality while uniquely producing structured canonical assets with explicit fur.
  • AnimalLift maintains strong LPIPS and MV-CLIP scores despite canonical reconstruction, while its lower Patch-FID indicates improved local fur realism.
  • The method achieves the best canonical geometry accuracy on the synthetic benchmark and an L1-Hair error of 0.34 cm at near real-world scale.Prior methods do not output fur, so they cannot be evaluated with the L1-Hair metric.
  • Ours and Ours-RandomPose perform closely, indicating that animation does not significantly degrade visual quality.
  • The canonical representation directly yields DCC-compatible assets that can be rigged, re-posed, groomed, simulated, and rendered.

5.5 Ablation Study

Ablations show that explicit fur, flow matching for texture, and classifier-free guidance materially affect local realism, while severe corruptions and viewpoint or pose changes cause only minor degradation.

  • Removing explicit fur increases Patch-FIDInc from 160.92 to 190.93, confirming the importance of explicit strand geometry for realistic fur appearance.
  • Replacing flow matching with direct regression causes the largest overall degradation and increases Patch-FIDInc to 210.32.The resulting textures are blurrier.
  • Disabling classifier-free guidance reduces texture sharpness and semantic consistency, as reflected by lower MV-CLIP and higher Patch-FIDInc.
  • Severe brightness, JPEG, noise, and motion-blur corruptions cause only minor overall degradation, with brightness shift producing the largest performance drop.Reconstructed textures retain plausible fur colors and clear grooming details despite corrupted inputs.
  • Across viewpoint and pose changes, geometry, texture, and fur remain coherent with only minor metric variation and no substantial degradation.

5.7 Generalization to Held-out Breeds and Identities

AnimalLift generalizes to held-out breed and appearance combinations by recombining learned shape and appearance priors while preserving distinguishing animal traits.

  • AnimalLift preserves ear shape, muzzle length, body proportion, and local fur density for unseen breeds rather than retrieving a fixed template.

5.8 User Study

A blind expert study found AnimalLift consistently preferred for visual quality and asset usability, while requiring less cleanup than competing methods.

  • The study used ten experts, five randomly selected animal cases, anonymized methods, and standardized Blender workflows to assess production readiness.
  • Experts gave AnimalLift 50/50 votes for both visual quality and mesh or asset usability.
  • AnimalLift required 1.7 hours of average cleanup for static rendering, compared with 3.8 hours for Hunyuan-3D 2.1 and 6.8 hours for Trellis 2.

6 Conclusion and Discussion

AnimalLift uses a shared UV-aligned canonical representation to connect single-image reconstruction with animation and graphics workflows. The framework supports structured appearance editing and fur-related downstream applications, while remaining bounded by topology, species, individuality, and single-view ambiguities.

  • Conclusion: The shared UV-aligned representation preserves dense correspondence and supports editing, animation, grooming, and simulation-compatible rendering workflows.Geometry, texture, and fur are modeled together in a canonical space intended for graphics-pipeline use.
  • Limitations: The method assumes shared topology and currently targets a family of quadrupeds, while single-image ambiguity makes occluded surfaces plausible rather than ground-truth predictions.Canonical grooming also favors editability over transient photo-specific states and may miss fine identity cues such as scars or broken strands.
  • Results: Reconstructions generalize across species while preserving geometry, identity, and coherent hair patterns under diverse natural-scene lighting.The visualizations emphasize fur detail and structure in rendered results.
  • Evaluation: Expert evaluations assess static-render cleanup, walking-animation preparation effort, and the quality attainable after one hour of cleanup.The supplied figure describes the evaluation dimensions but does not report rating values.
  • Dataset: The dataset uses shared UV layouts across diverse animal models and includes category distributions for training and evaluation.These visualizations show the structured data foundation underlying cross-animal reconstruction and transfer.
  • Applications: Fur geometry and texture can be transferred between animals while preserving each target's underlying body shape and identity.The transfer operation demonstrates separable appearance components within the structured representation.
Loading 2609.09513v1…