DreamHuman: Animatable 3D Avatars from Text

1 Introduction

• General-purpose text-to-3D methods offer limited control and can produce unrealistic proportions, missing limbs, or incorrect numbers of fingers.The paper says geometric and kinematic human priors can effectively resolve anthropometric consistency problems.Source: §1 · p. 1
• The authors describe DreamHuman as a potential tool for artists and 3D animators to automate complex parts of the design process.Source: §1 · p. 2
• DreamHuman generates animatable 3D humans from text without supervised text-to-3D data or image conditioning.The models can be placed in a variety of poses.Source: §1 · pp. 1–2
• DreamHuman incorporates 3D human body priors and multiple losses to regularize avatar generation and reposing across structure, appearance, and deformation.Source: §1 · p. 2
• Semantic zoom with refining prompts adds detail to perceptually important body regions, including the face and hands.Source: §1 · p. 2

Text-to-3D generation

• Text-to-3D methods such as DreamFields and DreamFusion generate static scenes, and human prompts can yield anatomical artifacts; DreamHuman adds 3D human priors.Source: §2 · p. 2

Text-to-3D human generation

• Text-to-motion methods generate motions as 3D coordinates or body-model parameters, but do not produce photorealistic results.Source: §2 · pp. 2, 4
• Unlike AvatarCLIP's fixed skinning weights, DreamHuman learns per-instance pose-specific deformations that support loose garments such as skirts and dresses.Source: §2 · p. 4

Deformable Neural Radiance Fields

• Compared with H-NeRF's video supervision, DreamHuman uses text alone and is not constrained by the poses and viewpoints in a video.The paper states that this lets DreamHuman generalize better across poses and camera viewpoints.Source: §2 · p. 4

3.1 Architecture

• An imGHUM semantic-space NeRF models pose-dependent clothing deformations beyond the body model, supporting different human poses and shapes.The density proxy encourages conformity to body geometry and helps avoid removing limbs or fine structures; semantic zoom renders six body regions with adjusted prompts to improve effective resolution and produce crisper textures and geometric detail.Source: §3.1 · p. 5

3.2 Loss functions

• Geometry, normal, and binary-mask losses preserve body detail, smooth surface normals while encouraging pose conditioning, and prevent translucent clothing or hair.The density loss compares NeRF density with an imGHUM-derived proxy; predicted normals also help the model learn pose-dependent deformations.Source: §3.2 · p. 6
• SDS supervises generation from modified semantic-zoom prompts, alongside orientation and proposal-weight losses in the full objective.The full objective combines these with mask, normal, geometry, and other losses.Source: §3.2 · p. 6

3.3 Optimization

• Optimization samples random poses to learn shape- and pose-dependent surface geometry and help separate the avatar from background objects, while optimizing shape parameters and varying camera settings.The imGHUM shape parameters are optimized because body shape may be described explicitly or implicitly in the caption.Source: §3.3 · p. 7

4 Experiments

• The experiments show generated human models with diverse body shapes, skin tones, and body compositions across different poses.Additional results are available in the Supplementary Material.Source: §4 · p. 7

4.1 Ablation Study

• Semantic zoom produces much higher-quality textures for both the body and face.Source: §4.1 · p. 7
• With pose-dependent deformations, skirts and shorts move more naturally; without them, skirt geometry struggles and floating artifacts appear around the legs.The authors hypothesize that the model leverages learned knowledge of clothing drape without video or a text-to-video diffusion loss.Source: §4.1 · pp. 7–8, Figure 5 · p. 8

4.2 Comparison with the state of the art

• Against DreamFusion, DreamHuman avoids frequent partial-body outputs, scene entanglement, implausible proportions, limb errors, and degenerate geometry attributed to viewpoint overfitting.Both methods were prompted for full-body subjects; Figure 6 shows rendered models and surface normals.Source: Figure 6 · p. 8, §4.2 · p. 8
• Compared with AvatarCLIP [14], DreamHuman produces better geometry and texture; AvatarCLIP stays close to body geometry, limiting loose clothing and accessories.AvatarCLIP's textures also show significant artifacts and do not match DreamHuman's realism in the examples tried.Source: §4.2 · pp. 8–9, Figure 7 · p. 9

• DreamHuman uses statistical 3D human-body models and recent 3D modeling and text-to-3D advances to generate animatable avatars without paired text-to-3D supervision, with photorealistic detailed geometry and large-margin state-of-the-art gains.Source: §5 · p. 9
• Without 3D training data, fine wrinkles may be represented in albedo rather than geometry; albedo-shading entanglement can bake in reflections and shadows.Diffusion-model compute also limits very high-resolution textures and hair-like geometric detail; clothing-animation realism may benefit from a video model.Source: §5 · p. 9
• Despite curated, diverse body-shape models intended to reduce bias, dependence on diffusion models and LLM text encoders pretrained on uncurated data leaves generation potentially bias-vulnerable.Source: §5 · p. 9
• People-generation may be misused for fake media; outputs are typically less realistic than 2D-generated counterparts, and the authors recommend prompt filtering and unsafe-render detection.Source: §5 · p. 9
• Diverse body shapes, appearance, skin color, and clothing may support large-scale synthetic human-task datasets and, in turn, models with fairer outcomes across groups.Source: §5 · p. 9
• DreamHuman could complement creative professionals’ work, boost productivity, and potentially democratize 3D content creation that currently requires specialized knowledge and expensive proprietary software.Source: §5 · p. 9