PaperlayerView exact revision on arXiv

Source-linked AI summary

DiGS-Avatar: Single-Image Animatable 3D Human Reconstruction via UV-Space Diffusion

Jiakun Li, Li Fang, Hao Zhu, Fei Hu, Long Ye, Yuan Zhang, Jinyao Yan

arXiv:2608.20759v1cs.CV

TL;DR

Single-image 3D human reconstruction is difficult because clothing complexity, pose diversity, and severe occlusions make the task ill-posed. DiGS-Avatar reformulates reconstruction as diffusion-based UV-latent completion with teacher-student supervision and semantic feature aggregation, achieving strong fidelity and zero-shot generalization while producing an animatable avatar in 0.71 seconds.

  • Problem

    Single-image 3D human reconstruction is an ill-posed problem involving complex clothing, diverse poses, and severe occlusions.

  • Method

    DiGS-Avatar completes a partial, surface-aligned UV latent using a 2D diffusion student supervised by a multi-view teacher, then injects semantic features to recover fine details.

  • Results

    0.71 seconds; DiGS-Avatar achieves state-of-the-art or highly competitive visual fidelity, zero-shot generalization, and substantially higher inference efficiency.

  • Takeaways & Limitations

    The UV-space formulation provides 3D consistency by design while avoiding inference-time multi-view generation and enabling fully animatable 3D Gaussian avatars from a single image.

  • Takeaways & Limitations

    Reliance on explicit SMPL-X topology can degrade reconstruction quality for extreme unconstrained poses and highly loose, non-rigid clothing.

Abstract

from arXiv · show

Single-image 3D human reconstruction often suffers from over-smoothed textures and geometric inconsistencies. While diffusion models improve generative quality, their reliance on multi-view synthesis prior to 3D reconstruction is computationally expensive and prone to view inconsistency. We propose DiGS-Avatar, which reformulates this task as an efficient, diffusion-based UV-latent completion task, ensuring 3D consistency by design. To capture accurate spatial structure, we introduce a teacher-student framework where a multi-view teacher provides geometrically aligned pseudo-ground-truth latents to supervise a single-view diffusion student. Treating this inferred latent as a robust structural skeleton, our method injects high-level semantic features to accurately recover fine textural details without disrupting spatial integrity. The refined representation is then decoded into 3D Gaussian primitives. Extensive experiments demonstrate that DiGS-Avatar achieves state-of-the-art or highly competitive visual fidelity and zero-shot generalization, while reconstructing a fully animatable 3D avatar in just 0.71 seconds. Code is available at https://github.com/KLMAV-CUC/DiGS-Avatar.

1 Introduction

DiGS-Avatar addresses the ambiguity and inconsistency of single-image 3D human reconstruction by completing diffusion-based latents in a canonical UV space. A teacher-student framework and semantic aggregation produce detailed, geometrically consistent, animatable avatars efficiently.

  • Single-image reconstruction is ill-posed because clothing complexity, pose diversity, and severe occlusions make high-fidelity animatable humans difficult to recover.
  • Diffusion methods improve generative capability but multi-view synthesis before reconstruction incurs high cost and can produce view inconsistencies.
  • DiGS-Avatar reformulates reconstruction as diffusion-based UV-latent completion in a canonical, surface-aligned domain, making 3D consistency intrinsic to the representation.Each UV pixel maps to a unique surface point, while the 2D formulation avoids computationally heavy 3D generation.
  • A multi-view teacher supplies geometry-aligned pseudo-ground-truth UV latents for a single-view diffusion student, establishing a structurally coherent latent.
  • GASA injects high-level semantic features into the inferred structural latent to recover fine details lost through VAE compression.
  • 0.71 seconds: DiGS-Avatar generates photorealistic, fully rigged animatable avatars while achieving state-of-the-art or highly competitive visual fidelity and zero-shot generalization.

2 Related Work

Prior work spans implicit and feed-forward reconstruction, diffusion-based refinement and view synthesis, and native 3D generative models. These approaches trade off detail, generalization, geometric consistency, and computational cost in different ways.

  • Implicit-surface methods such as PIFu and PIFuHD are computationally expensive and sensitive to SMPL estimation errors.
  • Feed-forward methods generate explicit 3D representations efficiently, but examples such as E3Gen and IDOL have constraints on arbitrary inputs or reconstruction detail.
  • Diffusion-based approaches improve appearance realism or use canonical reconstruction, but pixel-domain refinement can remain inefficient and may not correct geometric errors.
  • Native 3D generative models enforce geometric consistency but rely on computationally expensive volumetric processing and multi-stage pipelines.
  • DiGS-Avatar instead uses a UV-space diffusion framework whose pipeline includes partial-latent completion and decoding into 3D Gaussian primitives.

3 Method

DiGS-Avatar reconstructs a single-image human by completing a geometry-aligned UV latent with a diffusion student supervised by a multi-view teacher. Progressive semantic feature injection refines this structural representation before decoding it into an animatable 3D Gaussian avatar.

  • 3 Method: DiGS-Avatar reformulates reconstruction as diffusion-based completion in a canonical UV space where each UV pixel maps to a unique surface point.This design makes the 2D latent 3D-consistent by construction.
  • 3 Method: The multi-view teacher fuses aligned features into a complete UV latent that supervises the single-view student.Plücker ray embeddings encode camera origin and ray direction while the UV-Alignment Transformer resolves multiview inconsistencies.
  • 3 Method: The student maps visible single-view features into partial UV space and uses conditional 2D diffusion to infer missing regions from the teacher latent.The latent is noised during training, and the diffusion model predicts the noise needed to recover the clean latent.
  • 3 Method: GASA progressively injects deep, mid-level, and shallow appearance features at 32×32, 64×64, and 128×128 resolutions.Cross-attention enriches the structural skeleton with semantic and local texture cues while preserving geometry-aligned constraints.
  • 3 Method: The shared decoder converts the refined UV feature map into Gaussian attribute maps, and inference executes only the lightweight student path and decoder.The output is a canonical 3DGS avatar whose Gaussians can be animated through the pose-aware representation.

4 Experiments

Experiments show that DiGS-Avatar combines strong visual fidelity, zero-shot generalization, animation consistency, and substantially faster inference. Ablations support the contributions of teacher supervision, diffusion completion, GASA, and conditioning, while extreme non-rigid clothing remains a scope boundary.

  • Comparison with State-of-the-Art: 0.71s inference time makes DiGS-Avatar over 3× faster than IDOL and about 70× faster than SyncHuman.The method requires approximately 60 GPU-hours for training.
  • Zero-Shot Generalization and In-the-Wild Evaluation: On SIZER, DiGS-Avatar achieves the highest PSNR and SSIM, remains highly competitive in LPIPS, and preserves loose-garment geometry in qualitative comparisons.SIZER tests clothing-size and loose-garment variation outside the training distribution.
  • Animation and Novel Pose Synthesis: In-the-wild reconstructions handle diverse casual clothing, complex illumination, and poses, while reconstructed avatars retain coherent textures under novel poses and extreme articulations.The avatars are directly driven by standard LBS through their SMPL-X binding.
  • Ablation Study and Analysis: Teacher supervision, the adapted diffusion prior, denoising loss, GASA, and conditioning each materially improve reconstruction quality in ablation studies.Removing the teacher, diffusion components, GASA, or conditioning lowers reported reconstruction quality and weakens texture or alignment.
  • Ablation Study and Analysis: Extremely loose or non-rigid clothing can exceed the offset capacity of Gaussian primitives because the representation remains anchored to SMPL-X topology.The paper identifies this as a limitation and discusses related failure cases in supplementary material.

5 Conclusion and Limitations

DiGS-Avatar combines 2D UV-latent completion, multi-view teacher-student distillation, and semantic aggregation to generate photorealistic, animatable 3D Gaussian avatars efficiently. Its main limitation is reliance on explicit SMPL-X topology, which can degrade reconstruction for extreme poses and highly loose, non-rigid clothing.

  • Conclusion: DiGS-Avatar reformulates single-image reconstruction as 2D UV-latent completion and combines teacher-student distillation with semantic GASA aggregation.The resulting representation is decoded into photorealistic, instantly animatable 3D Gaussian avatars.
  • Conclusion: Extensive evaluations report state-of-the-art fidelity, robust zero-shot generalization, and exceptional computational efficiency.
  • Limitations: The method's explicit SMPL-X topology constrains its geometric space and can degrade quality for extreme, unconstrained poses or highly loose, non-rigid clothing.Future work is directed toward relaxing these topological constraints and exploring higher-resolution latents.

A Architectural Details

The architecture encodes images and appearance features, aligns multi-view information into a canonical UV representation, completes UV latents with diffusion, aggregates semantics, and decodes the result into Gaussian attributes.

  • VAE Encoder: The VAE encoder resizes images to 1024 × 1024 and converts them into compact 32 × 32 × 32 latent tensors with 32 channels.These tensors define the shared representation space for teacher and student networks.
  • Appearance Encoder: The DINOv3 ViT-B/16 appearance encoder produces 64 × 64 feature maps with 768-dimensional tokens that preserve semantic and fine-appearance cues.
  • UV-Alignment Transformer: The UV-Alignment Transformer combines per-view features, Plücker ray embeddings, and 1,024 learnable UV query tokens through four transformer blocks.The query tokens integrate geometrically consistent information across views before forming the canonical UV feature map.
  • Latent Diffusion Model: The latent diffusion model uses the SANA-1.6B Linear DiT backbone to perform diffusion-based UV-latent completion.
  • GASA and UV-to-3DGS Decoder: GASA injects high-level semantic cues into a geometrically structured latent, while the UV-to-3DGS decoder converts enriched tokens into dense Gaussian attribute maps.The student distills geometry-aligned structure from multi-view supervision, and the decoder uses hierarchical convolutional upsampling.

B Detailed Baseline Configurations and Fairness Justification

The evaluation uses official pretrained baselines, recommended inference pipelines, shared camera and SMPL-X inputs, and training data choices justified by the teacher's multi-view supervision requirements.

  • Fair Evaluation: All baselines use officially released pretrained weights and recommended inference pipelines for fair comparison.The evaluated methods are IDOL, LHM, SIFU, Human3Diffusion, TRELLIS, and SyncHuman.
  • Fair Evaluation: Every dataset evaluation provides all methods with identical ground-truth camera parameters and ground-truth SMPL-X fits.The datasets are HuGe100K, THuman 2.1, 2K2K, and SIZER.
  • Training Configuration: DiGS-Avatar additionally uses 2K2K for final fine-tuning because its multi-view teacher requires geometrically consistent multi-view supervision.The authors note that diffusion-generated HuGe100K views may contain cross-view inconsistencies that harm teacher latent construction.

C Additional Qualitative Results

Additional results evaluate DiGS-Avatar on unconstrained in-the-wild images containing difficult illumination, occlusions, and diverse casual garments.

  • Additional Qualitative Results: The in-the-wild evaluation includes everyday photographs with complex illumination, heavy occlusions, and diverse loose garments.Examples include flowing coats, multi-layered dresses, and loose-fitting sportswear.
  • Additional Qualitative Results: Using Multi-HMR for automatic SMPL-X parameter estimation, the framework evaluates reconstruction on uncalibrated real-world images.

C.2 Animation and Temporal Stability

DiGS-Avatar produces avatars that can be animated directly with novel pose sequences. Its canonical SMPL-X UV anchoring enables standard Linear Blend Skinning without test-time optimization or learned deformation fields.

  • C.2 Animation and Temporal Stability: Canonical SMPL-X UV anchoring makes the reconstructed 3D Gaussian primitives instantly animatable with standard Linear Blend Skinning.The approach bypasses computationally expensive test-time optimization and complex deformation-field learning.
  • C.2 Animation and Temporal Stability: Additional results visualize reconstructions under diverse clothing, poses, and illumination.
  • C.2 Animation and Temporal Stability: Additional frame-by-frame results show avatars driven by novel pose sequences.

D User Study

The user study compares DiGS-Avatar with IDOL and LHM on challenging single-view cases using human preferences for reconstruction and animation quality. Evaluators judged geometric consistency, texture clarity, animation quality, and overall preference.

  • D User Study: The study compares DiGS-Avatar, IDOL, and LHM across 50 challenging cases from dataset and in-the-wild domains.Thirty evaluators viewed reference images alongside randomized side-by-side novel views and animated sequences.
  • D User Study: N = 30 human evaluators selected preferred results across three criteria and an overall preference.
  • D User Study: The criteria were geometric consistency, texture clarity, and animation quality.They covered structural accuracy, high-frequency appearance details, temporal coherence, and animation artifacts.
  • D User Study: Table S1 reports the percentage of evaluations preferring DiGS-Avatar over IDOL and LHM across the 50 cases.

E Sensitivity to SMPL-X Estimation and Failure Cases

DiGS-Avatar’s performance depends on accurate SMPL-X pose and shape estimates, which provide its geometric alignment. Errors can cause feature misalignment, blurred identity details, mean-shaped reconstructions, or failures on extreme non-rigid clothing.

  • E Sensitivity to SMPL-X Estimation and Failure Cases: Performance is bounded by the accuracy of the SMPL-X pose and shape estimates used as the underlying topological prior.Inaccurate estimates can produce feature misalignment and identity degradation despite avoiding severe tearing.
  • E Sensitivity to SMPL-X Estimation and Failure Cases: Incorrect SMPL-X estimates force UV-latent sampling from wrong image locations, causing mismatched clothing attributes and blurred facial or hairline details.
  • E Sensitivity to SMPL-X Estimation and Failure Cases: Severe pose failures remove reliable geometric conditioning, causing the diffusion model to generate an average face and generic body shape.Visible discrepancies can appear in body volume, limb angles, and limb proportions.
  • E Sensitivity to SMPL-X Estimation and Failure Cases: Explicit 3D positional offsets let Gaussian primitives deviate from the bare-body surface for standard clothing, but fail for extreme non-rigid topologies.Exceptionally loose, floor-length dresses may yield flattened textures or clipped geometry.
  • E Sensitivity to SMPL-X Estimation and Failure Cases: Future work targets relaxed topological constraints and greater robustness of the structural latent to initial pose errors.
Loading 2608.20759v1…