PaperlayerView exact revision on arXiv

Source-linked AI summary

Efficient 3D Gaussian Head Avatars for Edge Devices

Umar Farooq, Jean-Yves Guillemaut, Adrian Hilton, Marco Volino

arXiv:2610.09821v1cs.CV

TL;DR

Studio capture of high-quality human avatars is expensive and time-consuming, while generated avatars can lack fine geometric detail and multi-view consistency. The paper proposes an efficient, style-conditioned generator and reports substantial reductions in model complexity while maintaining competitive generation quality.

  • Problem

    Studio capture of high-quality human avatars is expensive and time-consuming, and generated avatars can lack fine geometric detail and multi-view consistency despite their broad applications.

    Source: §1 · p. 1

  • Method

    The authors introduce a style-conditioned generator using depth-wise separable convolutions, reformulated modulation and demodulation, and a parameter-efficient Gaussian attribute output block.

    Source: §1 · p. 2

  • Results

    The approach reduces parameter count by 70%, computational cost by 94%, and model size by up to 81% compared with the baseline while retaining competitive generation quality.

    Source: §5 · p. 8

  • Takeaways & Limitations

    The approach enables synthesis of new 3D Gaussian head avatars on CPUs and mobile devices through a web browser without dedicated GPU acceleration or application-specific software.

    Source: §5 · p. 8

    Some parameters are more sensitive to quantisation than others, so quality-aware quantisation is left for future work.

    Source: §4.1 · p. 6

Abstract

from arXiv · show

Generative 3D Gaussian head avatars provide high-quality, efficient rendering, but synthesising the Gaussian representation remains computationally expensive, limiting deployment on resource-constrained and edge devices. We introduce an efficient generator architecture for unconditional 3D Gaussian head synthesis, based on a parameter-efficient synthesis block and depth-wise separable convolutions while retaining style-based conditioning. Our architecture reduces generator complexity without requiring model compression or quantisation. Compared with the baseline model, our approach reduces FLOPs by 94%, parameter count by 70%, and model size by 81%, while maintaining competitive generation quality. We further demonstrate practical CPU inference and browser-based execution on mobile devices using ONNX Runtime, enabling 3D Gaussian avatar synthesis without dedicated GPU hardware or application-specific software. In addition to conventional image-quality metrics, we evaluate multi-view consistency, training cost, and deployment performance. Code, trained models, and evaluation tools will be released publicly.

1. Introduction

Automated avatar generation addresses the cost of studio capture, but generating Gaussian representations remains expensive even as 3D Gaussian Splatting speeds rendering. The paper proposes a style-conditioned, efficient generator for unconditional 3D Gaussian head synthesis and downstream use.

1. Introduction

  • Existing automated pipelines generate avatars from images or video, while learned priors rely on large datasets and generated avatars can lack fine geometry and multi-view consistency.The introduction also notes extensive GAN exploration for random-identity head avatars.Source: §1 · p. 1
  • The overview depicts generating a novel 3D Gaussian avatar from a random latent code, followed by support for animation and latent manipulation.The broader motivation includes applications in creative industries, gaming, telecommunications, AR/VR, and telepresence.Source: Figure 1 · p. 1, §1 · p. 1
  • 3D Gaussian Splatting enables a differentiable, explicit representation in generative training, and prior Gaussian head avatars render quickly at training and inference time.Follow-up work improved multi-view consistency and background separation and introduced animation capabilities.Source: §1 · p. 1
  • The Gaussian-generation step remains computationally expensive and dominates unconditional synthesis, limiting the full pipeline on resource-constrained and edge devices.A lightweight generator could support novel identities on-device while retaining style-based runtime modification.Source: §1 · pp. 1–2
  • The proposed generator combines depth-wise separable synthesis blocks, reformulated modulation and demodulation, and a parameter-efficient Gaussian attribute output block while retaining style-based conditioning.The authors report reductions of up to 70% in parameters and 94% in FLOPs while retaining the original Gaussian output resolution and primitive count.Source: §1 · p. 2

2. Related Work

Related work spans 3D-aware head generation, avatar animation, efficient generators, Gaussian compression, and on-device inference. This paper focuses on the cost of synthesizing new Gaussian head identities, complementing work on animation and post-optimization compression.

3D Aware Generative Head Models

  • Prior 3D-aware head generators use NeRF, GANs, or diffusion; later work explores geometry representations, full-head coverage, expression control, and multi-view generation.NeRF-based approaches and multi-view diffusion are described as computationally costly, and multi-view diffusion also has consistency limitations.Source: §2 · p. 2
  • Gaussian head-avatar work includes texture-space, hierarchical, and template-UV generators, with methods addressing consistency, editability, and composable hair and face generation.Across these approaches, generator forward-pass efficiency remains relatively underexplored.Source: §2 · p. 2

Animatable Gaussian Head Avatars

  • Animation methods bind or deform Gaussian primitives, while AGORA [13] reduces runtime deformation cost with a smaller MLP; this work instead targets synthesis of previously unseen identities.The paper characterizes its synthesis focus as complementary to deformation and animation approaches.Source: §2 · p. 2

Efficient Generators and GAN Architectures

  • Efficient GAN work uses pruning, variable operating points, or altered generator architectures; MobileStyleGAN and BlazeStyleGAN produce RGB rather than structured multi-channel Gaussian outputs.The paper distinguishes its Gaussian-output focus from these RGB-output generators.Source: §2 · p. 2

Efficient Gaussian Representations

  • Gaussian compression methods reduce primitive storage after optimization, making them complementary to reducing the cost of generating the primitives.The cited approaches include quantization and entropy coding, reducing spherical-harmonic degree, or organizing primitives into a grid for image codecs.Source: §2 · p. 3

On-Device Inference

  • Prior on-device work adapts neural fields or diffusion models for mobile execution, while AGORA [13] speeds edge-device animation without addressing initial identity synthesis.Browser-based inference can be substantially slower than native execution on the same mobile hardware [49].Source: §2 · p. 3

3. Method

The method builds a StyleGAN2-based UV-map generator for Gaussian attributes, adapting style conditioning to depth-wise separable convolutions. Reduced-width blocks and position and scale regularization complete the design.

3.1. Preliminaries

  • 3D Gaussian attributes occupy UV maps, while FLAME initializes sampled Gaussians near the mesh and softly regularizes geometry across views and poses.Pixel color is alpha-blended from ordered Gaussians, with view-dependent appearance represented by spherical harmonics.Source: §3.1 · pp. 3–4

3.2. Architecture Overview

  • Style modulation moves to instance-normalized inputs before depth-wise filtering, while demodulation follows point-wise convolution, preserving style dependence in the composed operation.The depth-wise style magnitude would otherwise cancel during demodulation.Source: §3.2 · pp. 4–5, Figure 3 · p. 4

3.3. Generator Design

  • Intermediate feature widths shrink to one-quarter for the static model and one-half for the main animatable model; the static model caps layers at 256 channels and trains its final four blocks in FP16.Gaussian attributes remain at the original spatial resolution so image resolution and Gaussian count are unchanged.Source: §3.3 · p. 5

3.4. Loss Function

  • Training uses the non-saturating GAN loss with position and scale regularization; the regularizers apply to predicted position offsets and scales.The discriminator and generator operate with a renderer, camera pose, and FLAME expression code.Source: §3.4 · p. 5

4. Results and Evaluation

The evaluation reports substantial generator savings, practical CPU and browser-based inference, and ablations showing that efficiency gains depend on architectural choices rather than capacity reduction alone. Qualitative results also show latent interpolation and expression-conditioned animation.

4. Results and Evaluation

  • Training first uses 6.5M images and then 15M additional images; ablations use 5M images, and FID is calculated on PyTorch checkpoints.Evaluation uses FFHQ, with FID, FLOPs, latency, and file size as primary metrics; FID3D and multi-view metrics assess additional quality dimensions.Source: §4 · pp. 5–6

4.1. Quantitative Comparison

  • CPU inference latency falls by 54% versus the baseline, and the full unconditional synthesis pipeline runs in 148 ms on CPU.ONNX Runtime Web also runs on a Pixel 10 without GPU acceleration or runtime-specific optimisation; operator decomposition and upsampling overhead partly explain why latency falls less than FLOPs.Source: §4.1 · p. 6
  • The method reduces parameter count by 70%, FLOPs by 94%, and model storage size by 81% versus the baseline while maintaining competitive FID.Table 1 compares generation quality and computational and deployment costs at 512² rasterisation; the reported model weights are uncompressed and unquantised.Source: §4.1 · p. 6, Table 1 · p. 7
  • FID3D increases from 11.57 to 16.40, a discrepancy the authors say suggests metrics capture different aspects of multi-view generation and may point to oversmoothing in their method.The experiment runs for 5M images; the full run on the larger model reduces overall training time by 42%.Source: §4.1 · p. 7

4.2. Ablations

  • Depth-wise separable blocks retain baseline FID of 9.6 while reducing parameters from 28M to 8M, whereas naive resolution or channel reductions raise FID to 17.2 and 17.6.A narrower configuration with spherical harmonics reduced to SH0 reaches FID 11.1, 4M parameters, and 197 ms latency; further resolution reduction reaches 2.7G FLOPs and 148 ms but raises FID to 15.8.Source: §4.2 · pp. 7–8, Table 3 · p. 8

4.3. Qualitative Results

  • Latent-code interpolation produces smooth identity transitions, and the animation model generates FLAME-conditioned expression changes including independent eye and jaw motion.The supplementary results also include animation driven by FLAME parameters estimated from monocular video.Source: §4.3 · p. 8, Figure 6 · p. 8

5. Conclusion

The proposed generator reduces the cost of 3D Gaussian head synthesis while retaining competitive quality, and the authors demonstrate CPU and mobile-browser execution. The conclusion also identifies quality, deployment, and ethical considerations.

5. Conclusion

  • 70% fewer parameters, 94% lower computational cost, and up to 81% smaller model size than the baseline accompany competitive generation quality and CPU/mobile-browser synthesis.The synthesis runs without dedicated GPU acceleration or application-specific software.Source: §5 · p. 8
  • The authors flag smaller latency gains than compute reductions, a PSNRMV/SSIMMV versus FID3D discrepancy, reduced view-dependent appearance with fewer spherical harmonics, and no WebGPU testing.They say the metric discrepancy requires further investigation.Source: §5 · p. 8
  • The authors warn that FFHQ demographic biases may appear in generated-avatar distribution and quality, and that easier creation brings dual-use risks such as misinformation and deepfakes.They encourage appropriate safeguards alongside continued research.Source: §5 · p. 9

A.1. Training and Measurement Protocol

  • FID and FID3D are computed on PyTorch checkpoints, not exported ONNX checkpoints; the authors report near-identical outputs between them.CPU measurements use ONNX Runtime on an AMD EPYC 9535, restricted to four threads and cores; reported results use the best repeated run after warm-up.Source: §A.1 · p. 11

A.2. Computational Cost Evaluation

The edge-device evaluation runs the generation pipeline and animation MLP separately on a Pixel 10, with inference and evaluation performed on the phone through a web page.

A.2. Computational Cost Evaluation

  • The full generation pipeline and animation MLP are benchmarked individually on a Pixel 10 using ONNX Runtime, with inference and evaluation performed entirely on the phone.The model is served inside a simple web page.Source: §A.2 · p. 11
  • Transferring the complete model and dependencies totals 75.59 MiB across 24 files; cached files make subsequent application restarts considerably faster.The passage describes transfer as quick over a stable internet connection.Source: §A.2 · p. 11
  • Reported parameter and FLOP counts cover only the exported generator, excluding the inference-unused discriminator and the Gaussian rasteriser outside the exported graph.FLOPs are reported as Forward MACs, with one MAC defined as float(a) * float(b) + float(c).Source: §A.2 · p. 11

B. Architecture for Full Resolution

The 512² configuration changes rasterisation resolution rather than the Gaussian-producing network, so generator costs remain those of the 256² model. The appendix reports its generator cost profile and 256² generation quality.

B. Architecture for Full Resolution

  • At 512², rasterisation resolution changes but the generator graph, parameter count, FLOPs, and exported file size remain identical to the 256² model.The additional 512² rendering cost lies in the native CUDA Gaussian rasteriser, outside the exported graph.Source: B. Architecture for Full Resolution · p. 11
  • 10,673,119 parameters, 12.735 GFLOP (6.367 GMAC), and 46.116 MB characterize the generator; full-pipeline latency is 171.3 ms with four threads.The deformation branch accounts for roughly a fifth of parameters but a substantially larger share of FLOPs because it operates on the full deformation plane.Source: Table 4 · p. 11, B. Architecture for Full Resolution · p. 11
  • FID is 4.3 for our model versus 3.8 for AGORA [13] at 256² resolution.The authors describe the quality gap as modest relative to the reduced parameter, FLOP, and storage budget.Source: B. Architecture for Full Resolution · p. 11, Table 5 · p. 12

C. Architecture Details

The architecture details examine depth-wise convolutions, spherical-harmonic and precision choices, feature-map sizes, cumulative output heads, and quantisation sensitivity. The reported trade-offs include parameter savings, quality changes, and limits on precision reduction.

C.1. Depth-wise Separable Convolutions

  • Depth-wise replacements in synthesis and UV-channel layers significantly reduce parameter count while producing minimal change in FID.The experiments compare replacing synthesis-layer convolutions alone with replacing those and the UV-channel synthesis-layer convolutions.Source: §C.1 · p. 12

C.2. Spherical Harmonics, Channel Width and Precision

  • Halving channel width and height makes the model 36% faster than the baseline but worsens FID; FP16 in the last four synthesis blocks slightly increases inference time.The authors suggest unsupported FP16 operations and converting UV maps back to FP32 may contribute to the latency increase.Source: §C.2 · p. 12

C.3. Generator and Feature Map Sizes

  • Different feature-map sizes reduce parameters and FLOPs while keeping the Gaussian count fixed, so variant comparisons reflect generator capacity rather than primitive count.Upsampling before the Gaussian output layer maintains a consistent UV-map resolution.Source: §C.3 · p. 12

C.4. Generator Output Heads

  • Removing cumulative Gaussian output channels harms quality and fidelity; the authors hypothesize that lost direct gradients destabilize training because position errors are amplified by the renderer.A position error displaces a 3D primitive, unlike an RGB error that perturbs a pixel value.Source: §C.4 · p. 12

D. Quantisation Sensitivity

  • Uniform post-training quantisation trades substantial quality for additional size reduction; position and scale heads degrade quickly, motivating future per-attribute precision.The reported file sizes are uncompressed, unquantised FP32 weights and therefore an upper bound on storage cost.Source: D. Quantisation Sensitivity · p. 12

E. ONNX Export Ablation

The ONNX Runtime implementation closely reproduces the PyTorch output, and the exported model is equivalent to the one evaluated for quality. Mobile rendering uses a different renderer and introduces minor visual artifacts.

E. ONNX Export Ablation

  • ONNX Runtime closely replicates PyTorch output with minimal pixel differences, supporting that the efficiency measurements use a model equivalent to the one evaluated for quality.The conversion reimplements some optimized operations, including modconv, with standard supported operations; residual differences are concentrated at high-frequency edges and consistent with floating-point differences.Source: E. ONNX Export Ablation · pp. 12–13
  • On mobile, a web-based splat renderer replaces the quality-evaluation renderer and introduces minor visual artifacts absent from PyTorch results.The ONNX model produces UV maps, while rendering for quality evaluation uses the original Python and PyTorch codebase with the FLAME template mesh.Source: E. ONNX Export Ablation · p. 13

F. Additional Qualitative Results

At 2562 resolution, the proposed model produces identities of comparable visual quality to GGHead after both models are trained for 5M images. Its latent directions yield smooth, mostly disentangled edits while preserving identity, with appearance factors more cleanly separated than geometry.

F. Additional Qualitative Results

  • At 2562 resolution after 5M images of training, the proposed model produces identities of comparable visual quality to baseline GGHead, with stable geometry across sampled identities.Figure 7 presents the qualitative comparison.Source: F. Additional Qualitative Results · p. 13

G.1. Latent Space Variation

  • Traversing one intermediate-latent direction changes a single factor smoothly while largely preserving identity, retaining editable style space without inference-time optimisation.Each edit requires one additional generator forward pass; appearance directions are more cleanly separated than geometry directions.Source: §G.1 · p. 13
  • The observed smooth, largely disentangled traversals indicate that the reduced-capacity generator has not collapsed onto a small set of modes.The mapping from latent space to Gaussian attribute maps remains locally well behaved.Source: §G.1 · p. 13

G.2. Gaussian Attribute Sensitivity

Attribute groups differ in their resolution requirements: position is most sensitive, while opacity and spherical-harmonic groups tolerate substantial reduction. The findings motivate allocating resolution and precision by attribute group rather than uniformly.

G.2. Gaussian Attribute Sensitivity

  • The sensitivity test reduces one attribute group at a time, bilinearly restores it, leaves the other five groups untouched, and measures FID3D on frozen weights at inference.The intervention estimates each group's marginal value of resolution.Source: §G.2 · p. 13, Figure 9 · p. 14
  • Position channels are by far the most resolution-sensitive: reducing their resolution degrades FID3D before any other group is affected.Low-pass filtering positions moves neighboring primitives together and removes geometric detail that the renderer cannot recover.Source: §G.2 · p. 13, Figure 9 · p. 14
  • Opacity and spherical-harmonic groups tolerate substantial resolution reduction before their FID3D curves depart from the unmodified model.Scale and rotation follow position but show a flatter response because these attributes vary more slowly across the head surface.Source: §G.2 · p. 13, Figure 9 · p. 14
  • The curve ordering matches the reported quantisation sensitivity, with position and scale least tolerant; geometric attributes carry the representation's information budget.Spending resolution and precision per attribute group rather than uniformly is identified as a promising direction for further efficiency gains.Source: §G.2 · p. 14

H. Additional Animation Results

Additional animation results show expression-conditioned deformations from FLAME parameters, including independent eye and jaw motion, as well as avatars driven by FLAME parameters estimated from monocular video.

H. Additional Animation Results

Loading 2610.09821v1…