Source-linked AI summary
Revisiting Avatar-As-Image: High-Fidelity Registration is All You Need
Margaret Kostyrko, Yuxuan Xue, Garvita Tiwari, Gerard Pons-Moll
TL;DR
AvaImg addresses the lack of high-fidelity SMPL(−X)+D registrations with UV texture for arbitrary clothed scans, a prerequisite for reliable Avatar-As-Image modeling. Its multi-stage pipeline combines signed winding-number body fitting, efficient optimization, and coarse-to-fine displacement recovery, outperforming baselines across six datasets while producing image-compatible UV maps.
Problem
Prior UV-based clothed-human methods lacked the high-fidelity registration needed to align scan geometry and appearance into reliable standardized maps.
Method
AvaImg uses multi-stage optimization with signed winding numbers for body-inside-clothing fitting, an efficiency cascade, and coarse-to-fine displacement optimization.
Results
AvaImg outperforms all baselines across body fitting, shape estimation, and surface registration on six datasets, with textured registrations reaching PSNR=34.48dB.
Takeaways & Limitations
Frozen-VAE encode-decode with only 0.76mm added Chamfer error supports using AvaImg’s UV maps with pretrained image generative models.
Takeaways & Limitations
The SMPL(−X)+D representation is limited for loose or decoupled garments, self-contact can cause texture bleeding, and processing takes approximately 25 minutes per scan.
Abstract
from arXiv · showhide
The representation of 3D clothed humans as standardized 2D UV texture and displacement maps over an underlying body model has long been studied. This compact representation is enticing as it enables pretrained image networks to process, generate, and edit 3D avatars, but is only useful if scans are accurately aligned and brought into correspondence via high-fidelity registration. This prerequisite has never been met, which we argue explains the limited quality of prior UV-based methods for clothed humans. Despite its significance, no public method produces high-fidelity SMPL(-X)+D registrations with UV texture from arbitrary clothed scans. We present AvaImg, a multi-stage optimization pipeline, to close this gap: it enforces body-inside-clothing constraint via signed winding numbers, made viable by a three-level efficiency cascade (~10x runtime reduced, ~95% storage saved), and recovers fine surface detail using coarse-to-fine displacement optimization. AvaImg outperforms all baselines in body fitting, shape estimation, and surface registration across six datasets, yielding textured registrations near-indistinguishable from scans (PSNR=34.48dB). For validation of AvaImg's Avatar-as-Image representation as imminently compatible with image foundation models, we auto-encode our UV maps via the frozen FLUX VAE. This achieves only 0.76mm added Chamfer error relative to scan and shows that the resulting maps lie within natural-image distributions, supporting the use of 2D generative priors for 3D avatar generation. Code, data, and Singularity containers will be at https://yuxuan-xue.com/avaimg.
1 Introduction
AvaImg addresses the missing high-fidelity registration needed to make standardized UV representations of clothed humans useful for image-based processing and generation. Its pipeline combines robust body-inside-clothing fitting, efficient optimization, and fine-detail recovery to produce UV-ready registrations.
- Motivation: High-fidelity registration is the prerequisite for using standardized UV texture and displacement maps with 2D generative models.Shared SMPL(−X) UV coordinates make each pixel correspond to the same anatomical location across subjects, but prior UV methods suffered from registration artifacts.
- Registration gap: Existing public tools and dataset registrations lack the required combination of SMPL−X support, UV texture mapping, and reliable body-clothing separation.RMR lacks SMPL−X and UV mapping, while established dataset registrations show pervasive body-clothing interpenetration.
- AvaImg: AvaImg combines multi-view pose initialization, signed-winding-number body fitting, coarse-to-fine displacement optimization, and UV remapping.The approach replaces surface-normal sidedness heuristics with volumetric containment and handles noisy or open scan surfaces.
- Efficiency: ∼10× lower runtime and ∼95% lower storage make signed winding constraints practical at dataset scale.A three-level efficiency cascade enables the volumetric fitting constraint to operate on large scan collections.
- Validation: Frozen diffusion-VAE encode-decode confirms that AvaImg’s UV texture and displacement maps lie within natural-image distributions.This validation supports compatibility with pretrained image generative models without requiring specialized 3D architectures.
2 Related Work
Prior work established parametric body models and UV representations for clothed humans, but registration quality remains the central bottleneck. AvaImg targets this bottleneck while combining body fitting, surface registration, and texture mapping in one pipeline.
- 2.1 Clothed Human Registration: Body-fitting methods range from optimization-based SMPLify variants to faster learned approaches, but body-centric methods do not capture clothing surfaces.The related-work landscape separates naked-body recovery from explicit exterior clothing reconstruction.
- 2.1 Clothed Human Registration: AvaImg introduces conjoined capacity for body fitting, surface registration, and texture mapping.This combines capabilities that prior registration and UV-space approaches treat separately.
- 2.2 2D Representations for 3D Humans: The method overview connects clothed-scan registration to image-compatible UV outputs through four stages: pose lifting, physics-aware fitting, displacement refinement, and remapping.The staged design links registration improvements directly to the Avatar-As-Image representation.
- 2.2 2D Representations for 3D Humans: UV-space methods can represent clothed-human texture and geometry as standardized images, but their quality depends on accurate registration.Without high-fidelity SMPL(−X)+D meshes, UV maps inherit geometric artifacts and texture misalignment.
3 Method
AvaImg registers a clothed scan to SMPL(−X)+D by initializing pose from scan evidence, fitting the naked body inside clothing, refining the exterior surface, and exporting aligned UV maps. Coarse-to-fine optimization and the SMPL(−X) manifold prior provide robustness to difficult scan geometry.
- Pipeline: AvaImg converts a raw scan into an SMPL(−X)+D registration with UV texture and displacement maps by solving pose, body fitting, and clothing-surface capture together.The output uses shared UV coordinates so texture and geometry form a standardized Avatar-As-Image representation.
- Pose initialization: Multi-view keypoint lifting provides scan-specific pose initialization for configurations where mean-pose initialization can reach local minima.The scan is rendered from 72 views across three elevation levels with full azimuth coverage before pose optimization.
- Body fitting: Signed winding numbers enforce the physical constraint that the naked body remains inside clothing while penalizing escaped body vertices more strongly.The robust Geman-McClure term and asymmetric pReLU weighting create a differentiable barrier against penetration.
- Body fitting: A three-level efficiency cascade makes generalized winding-number fitting practical by reducing computation and storage for high-polygon scans.The cascade includes scan decimation, precomputed voxel grids, and winding-band reduction.
- Progressive refinement: Coarse-to-fine displacement optimization recovers clothing folds, wrinkles, hair, and other exterior details that body fitting alone cannot represent.Edge coupling, smoothness, posed-versus-canonical consistency, and region-dependent weights progressively refine the surface.
- Progressive refinement: AvaImg’s refined geometry frequently surpasses comparable baselines and remains reliable across variable scan quality by using SMPL(−X) as a manifold prior.This prior helps compensate for noise, missing regions, and incomplete scan geometry.
- Avatar-As-Image output: The final registrations are remapped into UV texture and displacement images that encode-decode through a frozen pretrained diffusion VAE without fine-tuning.This validates the resulting maps as image-compatible representations for generative-model training.
4 Experiments
Across six datasets, AvaImg is evaluated for body fitting, shape and surface registration, texture fidelity, VAE compatibility, and key design choices. It reduces penetration while preserving scan proximity, produces high-fidelity textured registrations, and retains geometry through a frozen FLUX VAE roundtrip.
- 4.2 Comparison with Dataset Ground Truth: 28.2% to 15.0% penetration rate and 6.4 mm to 3.4 mm penetration depth improve over dataset ground truth across five datasets, with comparable scan proximity.The comparison is summarized in Table 2 and illustrated by body fits that remain enclosed within clothing while avoiding bent feet and stiff finger articulation.
- 4.3 Comparison with State-of-the-Art: 19.8% penetration rate and 3.62 mm penetration depth give AvaImg the best body-fitting trade-off among compared methods while maintaining 9.47 mm scan proximity.RMR and PTF obtain lower proximity but nearly half their body vertices penetrate clothing; AvaImg also reaches 7.72 mm shape-estimation Chamfer distance on BuFF.
- 4.3 Comparison with State-of-the-Art: 34.48 dB multiview PSNR and 5.19 FID show that AvaImg’s textured registrations preserve scan appearance after UV remapping.The evaluation compares rendered registrations with original scans, including fabric patterns and color gradients; public baselines do not support comparable texture remapping.
- 4.4 Image-based Avatar Representation: 0.76 mm added Chamfer error in the frozen FLUX VAE roundtrip indicates minimal degradation of AvaImg’s UV texture and displacement maps.The Chamfer distance increases from 3.15 mm to 3.91 mm, while UV texture reaches 38.6 dB PSNR and displacement reconstruction has 4.98 mm RMSE.
- 4.5 Ablation Study: 40.7% to 19.8% penetration rate and 10.06 mm to 3.62 mm penetration depth in the ablation show that signed winding numbers prevent physically implausible body–clothing collisions.Scan proximity remains comparable, while shape-under-clothing Chamfer distance on BuFF drops from 11.95 mm to 7.72 mm.
- 4.5 Ablation Study: Decimation and winding-band reduction reduce a large scan’s winding computation from about 2,400 s to 240 s and accelerate body fitting from about 1,700 s to 240 s.The profiled acceleration components provide the efficiency basis for practical signed-winding computation.
5 Conclusion
AvaImg produces high-fidelity SMPL(−X)+D registrations with UV texture mapping from arbitrary clothed scans and outperforms existing methods across six datasets. Its remaining limitations arise from SMPL(−X)+D topology, self-contact and joint unposing, and a roughly 25-minute per-scan runtime.
- AvaImg consistently outperforms state-of-the-art methods across body fitting, shape estimation, and surface registration on six datasets, achieving textured registrations with PSNR=34.48dB.The resulting registrations are described as nearly indistinguishable from scans.
- AvaImg enables UV maps to be encoded and decoded through a frozen image diffusion VAE with only 0.76 mm increased Chamfer error.This supports the maps’ compatibility with natural-image distributions and the Avatar-As-Image paradigm.
- SMPL(−X)+D limits reconstruction for loose or decoupled garments and can cause proximity-based association errors in complex geometric regions.Self-contact may produce texture bleeding, while unposing bent joints can introduce topological indentations.
- The full pipeline takes approximately 25 minutes per scan, targeting one-time high-fidelity ground-truth generation rather than feed-forward inference.The authors position these outputs as training data for faster alternatives.