PaperlayerView exact revision on arXiv

Source-linked AI summary

Mobile-GS: Real-time Gaussian Splatting for Mobile Devices

Xiaobiao Du, Yida Wang, Kun Zhan, Xin Yu

arXiv:2603.11531v1cs.CV

TL;DR

Mobile deployment of 3D Gaussian Splatting is limited by depth sorting, computational cost, and storage demands. Mobile-GS removes sorting, enhances view-dependent rendering, and compresses the representation, achieving real-time, compact, and high-quality rendering on mobile devices.

  • Problem

    Depth sorting in traditional alpha blending remains a computational bottleneck that impedes real-time 3D Gaussian Splatting on resource-constrained mobile devices.

  • Method

    Mobile-GS combines depth-aware order-independent rendering, neural view-dependent enhancement, distillation, vector quantization, and pruning for mobile-efficient Gaussian Splatting.

  • Results

    Mobile-GS achieves real-time mobile rendering, high-quality novel-view synthesis comparable to 3DGS, and improved speed, quality, and storage over prior lightweight approaches.

  • Takeaways & Limitations

    Mobile-GS provides a compact, high-quality Gaussian Splatting approach suited to real-time rendering on mobile devices.

  • Takeaways & Limitations

    Order-independent rendering can introduce undesired transparency effects in spatially occluded or partially overlapped regions.

Abstract

from arXiv · show

3D Gaussian Splatting (3DGS) has emerged as a powerful representation for high-quality rendering across a wide range of applications.However, its high computational demands and large storage costs pose significant challenges for deployment on mobile devices. In this work, we propose a mobile-tailored real-time Gaussian Splatting method, dubbed Mobile-GS, enabling efficient inference of Gaussian Splatting on edge devices. Specifically, we first identify alpha blending as the primary computational bottleneck, since it relies on the time-consuming Gaussian depth sorting process. To solve this issue, we propose a depth-aware order-independent rendering scheme that eliminates the need for sorting, thereby substantially accelerating rendering. Although this order-independent rendering improves rendering speed, it may introduce transparency artifacts in regions with overlapping geometry due to the scarcity of rendering order. To address this problem, we propose a neural view-dependent enhancement strategy, enabling more accurate modeling of view-dependent effects conditioned on viewing direction, 3D Gaussian geometry, and appearance attributes. In this way, Mobile-GS can achieve both high-quality and real-time rendering. Furthermore, to facilitate deployment on memory-constrained mobile platforms, we also introduce first-order spherical harmonics distillation, a neural vector quantization technique, and a contribution-based pruning strategy to reduce the number of Gaussian primitives and compress the 3D Gaussian representation with the assistance of neural networks. Extensive experiments demonstrate that our proposed Mobile-GS achieves real-time rendering and compact model size while preserving high visual quality, making it well-suited for mobile applications.

1 INTRODUCTION

Mobile-GS is a mobile-tailored Gaussian Splatting method that targets the sorting bottleneck in traditional alpha blending while reducing representation size for memory-constrained devices. It combines order-independent rendering, neural enhancement, distillation, vector quantization, and pruning to achieve real-time, high-quality mobile rendering, reaching 116 FPS on a Snapdragon 8 Gen 3 GPU.

  • Background: 3D Gaussian Splatting represents scenes with anisotropic 3D Gaussian primitives for high-quality 3D reconstruction and novel-view synthesis.The introduction situates 3DGS alongside Neural Radiance Fields and applications including self-driving and relighting.
  • Problem: Traditional alpha blending requires near-to-far depth sorting, which the authors identify as the primary computational bottleneck.Existing lightweight methods improve efficiency through pruning and compact representations but retain this sorting-dependent rendering process.
  • Method: Mobile-GS introduces depth-aware order-independent rendering to circumvent computationally intensive sorting and enable real-time mobile inference.The method is explicitly tailored for mobile devices and also includes neural view-dependent enhancement to improve rendering quality.
  • Method: Mobile-GS compresses Gaussian representations through first-order spherical harmonic distillation, neural vector quantization, and contribution-based pruning.First-order SH parameters are distilled under a pretrained teacher model, while grouped Gaussian parameters are quantized using K-means to reduce storage and improve rendering speed.
  • Results: 116 FPS rendering speed is achieved on a mobile device with a Snapdragon 8 Gen 3 GPU, demonstrating real-time mobile rendering.Experiments also report high-quality, visually pleasing novel-view synthesis comparable to original 3DGS and consistent superiority over previous lightweight approaches.

2 RELATED WORK

Related work covers 3D Gaussian Splatting representations and rendering, order-independent transparency methods, and Gaussian compression and pruning. Existing approaches improve rendering or compactness, but the supplied passages note challenges from storage and inference delay on edge devices.

  • 3D Gaussian Splatting: 3DGS represents scenes with anisotropic 3D Gaussians and renders novel views using a tile-based differentiable rasterizer.The passage places 3DGS alongside numerous variants, including MVGS, which applies multi-view learning during optimization.
  • Order Independent Transparency: Traditional order-independent transparency uses depth peeling or A-buffers, while later approaches approximate compositing to avoid explicit sorting.These methods target accurate transmittance modeling for semi-transparent structures such as flames, smoke, and clouds.
  • Order Independent Transparency: Because addition is commutative, the summations in the rendering equation can be computed in any order, but existing methods remain unsuitable for edge devices due to storage and inference delay.The passage identifies wbg, σi, and βi as learnable parameters and di as depth.
  • Gaussian Compression and Pruning: Gaussian compression research has explored vector quantization and entropy encoding to reduce the storage cost of Gaussian representations.LocoGS further introduces locality-aware compression into compact local representations, according to the supplied passage.

3 METHODOLOGY

Mobile-GS removes depth sorting through depth-aware, view-conditioned order-independent rendering, then restores quality with neural view-dependent opacity enhancement. It also compresses and prunes Gaussian representations using spherical-harmonics distillation, neural vector quantization, and contribution-based pruning.

  • Rendering Insight: Depth-aware order-independent rendering eliminates depth sorting with learnable, view-conditioned weights based on Gaussian scale, position, and camera-relative depth.Inverse depth reduces distant Gaussians’ contributions, while larger-scale Gaussians receive greater weight, enabling efficient parallel accumulation.
  • Neural View-dependent Enhancement: Neural view-dependent opacity enhancement addresses transparency artifacts from order-free rendering by using an MLP to predict per-Gaussian visibility from geometric and appearance features.The model uses Gaussian position, scale, rotation, spherical harmonics, and camera direction, with view-dependent opacity dynamically suppressing transparency.
  • First-order Spherical Harmonics Distillation: First-order spherical-harmonics distillation projects a teacher’s directional radiance into a compact 3 × 4 coefficient representation and adds scale-invariant depth supervision.The depth target uses ε = 1e−8 for training stability, avoiding strict L1/L2 matching because teacher depth may be unreliable.
  • Neural Vector Quantization: Neural vector quantization decomposes per-Gaussian attributes and spherical-harmonics features into compact components reconstructed by lightweight neural networks.Diffuse and view-dependent components are modeled separately, and both decoding MLPs are quantized to 16-bit precision.
  • Contribution-based Pruning: Contribution-based pruning progressively removes Gaussians with consistently low opacity and scale by accumulating pruning votes across intervals, while training uses the original 3DGS L1 and LDSSIM rendering loss.This voting mechanism mitigates early-training fluctuations before permanently pruning low-contribution primitives.

4 EXPERIMENTS

Experiments show that Mobile-GS delivers high-quality novel-view synthesis with real-time, compact deployment on mobile devices. Ablations attribute its efficiency and fidelity to order-independent rendering, view-dependent enhancement, pruning, and neural vector quantization.

  • Overall Comparison: Mobile-GS is compared with eight state-of-the-art methods across Mip-NeRF 360, Tank&Temples, and Deep Blending.Table 1 reports quantitative comparisons and highlights the best lightweight 3DGS results.
  • Overall Comparison: Mobile-GS produces sharper, more consistent novel views comparable to 3DGS and improves view-dependent effects in complex geometry and lighting.The reported quantitative and qualitative results support high-quality synthesis under challenging scene conditions.
  • Evaluation on Mobile: On Snapdragon 8 Gen 3, Mobile-GS outperforms quantized 3DGS, Mini-Splatting, and SortFreeGS in rendering quality, speed, and storage.Its advantages are attributed to sorting-free rendering, simultaneous Gaussian processing, quantization, and pruning.
  • Ablation Study: Replacing order-independent rendering with alpha blending slightly improves PSNR but significantly reduces rendering speed, establishing its efficiency contribution.Removing view-dependent enhancement also causes visual degradation, while its inclusion addresses transparency artifacts from overlapping geometry and depth ambiguity.
  • Contribution-based Pruning: Joint pruning by opacity and scale removes redundant Gaussians while preserving rendering fidelity more effectively than either single-attribute criterion.The strategy reduces memory and computational overhead with minimal precision loss.
  • Neural Vector Quantization: Neural vector quantization uses K-means codebooks, with sizes from 26 to 212 trading off PSNR against storage cost.Very small codebooks drastically degrade PSNR, whereas excessively large codebooks require substantially more storage.

5 CONCLUSION · A LLM USAGE STATEMENT · B PRELIMINARY: 3D GAUSSIAN SPLATTING

Mobile-GS is presented as a mobile-specific real-time Gaussian Splatting method built from rendering, enhancement, distillation, quantization, and pruning techniques. The surrounding sections state the authors’ limited LLM use and summarize 3DGS representation and rendering fundamentals.

  • 5 CONCLUSION: Mobile-GS is proposed as the first Gaussian Splatting method specifically designed for real-time rendering on mobile devices.The method combines depth-aware order-independent rendering, neural view-dependent opacity enhancement, first-order spherical harmonics distillation, neural vector quantization, and contribution-based pruning.
  • 5 CONCLUSION: The proposed techniques jointly target computational bottlenecks and representation compactness for mobile Gaussian Splatting deployment.The supplied conclusion identifies rendering, enhancement, distillation, vector quantization, and pruning as the method’s core innovations.
  • A LLM USAGE STATEMENT: The authors used a Large Language Model solely for language polishing, not for novel ideas, analysis, or scientific content.Its stated uses were grammar refinement, readability improvement, and consistency of tone and style; the authors report the research design, implementation, and results as original contributions.
  • B PRELIMINARY: 3D GAUSSIAN SPLATTING: 3DGS represents scenes with anisotropic 3D Gaussians parameterized by (µ_i, Σ_i, o_i, Y_i) for high-quality real-time rendering.The mean specifies world-space position, while the covariance describes the Gaussian ellipsoid’s anisotropic spatial extent and orientation.
  • B PRELIMINARY: 3D GAUSSIAN SPLATTING: Rendering projects each Gaussian onto the image plane using a perspective camera model before depth sorting them from near to far.The sorted Gaussians are then rendered through alpha blending.
  • B PRELIMINARY: 3D GAUSSIAN SPLATTING: Gaussian Splatting provides a differentiable, compact, and efficient representation and rendering approach for complex scenes and neural rendering optimization.The formulation includes transmittance in alpha blending, positional offsets ∆x_i = x_i−µ_i, and N 3D Gaussians.

C ADDITIONAL IMPLEMENTATION DETAILS

The implementation combines lightweight neural modules with compression and pruning strategies tailored to efficient Gaussian Splatting deployment. These components model view-dependent appearance, reduce attribute storage, and remove redundant Gaussian primitives during training.

  • Neural View-dependent Enhancement: The view-dependent enhancement uses a three-layer MLP with 256, 128, and 64 neurons to predict opacity o and auxiliary feature ϕ from Gaussian primitives.A Sigmoid constrains opacity to [0, 1], while ReLU is used for the other activation pathway.
  • Neural Vector Quantization: Neural vector quantization partitions Gaussian attributes into five sub-vector groups, applies K-means clustering, Huffman coding, and compact group-specific MLP refinement.The coding and entropy-compression stages reduce storage redundancy before neural refinement.
  • Distribution-based Pruning: Distribution-based pruning uses Iprune = 1000, v = 0.6, and τ = 0.2, operating only during the initial 25k training iterations to remove marginal Gaussian primitives.Restricting pruning to early training preserves finer details during later stages.

D DISCUSSION AND LIMITATIONS · D.1 DIFFERENCE WITH SORTING-FREE METHODS

Mobile-GS is presented as a comprehensive and efficient sorting-free renderer that targets high-quality, real-time mobile deployment. It distinguishes itself through Gaussian scale-aware weighting and lightweight neural modeling of view-dependent effects to mitigate order ambiguity without substantial overhead.

  • D.1 DIFFERENCE WITH SORTING-FREE METHODS: Mobile-GS, SortFreeGS, and GES all use sorting-free rendering, but Mobile-GS is described as more comprehensive and efficient.The comparison considers PSNR, storage cost, and rendering FPS.
  • D.1 DIFFERENCE WITH SORTING-FREE METHODS: Mobile-GS consistently outperforms prior sorting-free methods in PSNR, storage cost, and rendering FPS.The passage does not provide the corresponding Table 8 values.
  • D.1 DIFFERENCE WITH SORTING-FREE METHODS: SortFreeGS* denotes a quantized version of SortFreeGS, whereas the original SortFreeGS lacks a quantization stage.This distinction affects the comparison of storage and rendering performance.
  • D.1 DIFFERENCE WITH SORTING-FREE METHODS: SortFreeGS modulates Gaussian contribution using depth but omits Gaussian scale, which Mobile-GS identifies as critical.The approaches therefore differ in how Gaussian weights are computed.
  • D.1 DIFFERENCE WITH SORTING-FREE METHODS: GES first renders a depth image with conventional volume rendering, then filters distant Gaussians before later sorting-free rendering.It compares Gaussian depths against the rendered depth map in this two-stage process.
  • D.1 DIFFERENCE WITH SORTING-FREE METHODS: Overlapping geometry creates order ambiguity for sorting-free methods and can produce transparency artifacts.This challenge arises because sorting-free rendering lacks a complete rendering order.
  • D.1 DIFFERENCE WITH SORTING-FREE METHODS: SortFreeGS addresses order ambiguity with additional spherical harmonics parameters for view-dependent opacity, but this adds significant overhead for mobile use.The passage characterizes this design as unfavorable for practical mobile deployment.
  • D.1 DIFFERENCE WITH SORTING-FREE METHODS: Mobile-GS instead enhances view-dependent effects through a learnable parameter ϕ predicted by a lightweight MLP.The MLP is conditioned on inputs described in the passage as including viewing direction, 3D Gaussian geometry, and appearance attributes.

D.2 DISCUSSION

Mobile-GS enables real-time Gaussian Splatting on mobile and resource-constrained platforms while preserving rendering quality. Its sorting-free rendering and neural compression components balance speed, visual fidelity, and storage footprint.

  • Core method: Mobile-GS achieves real-time rendering on mobile and resource-constrained platforms without significantly sacrificing rendering quality.The method replaces traditional alpha blending with depth-aware order-independent rendering that avoids sorting.
  • Rendering quality: Depth-aware order-independent rendering improves runtime efficiency, while neural view-dependent enhancement and spherical harmonics distillation maintain visual fidelity in complex scenes.The rendering scheme is sorting-free and is combined with neural view-dependent enhancements and distillation.
  • Experimental findings: Mobile-GS balances rendering speed, storage footprint, and visual quality, outperforming existing lightweight Gaussian Splatting methods across multiple benchmarks.The reported effectiveness includes depth-aware order-independent rendering, neural view-dependent enhancement, spherical harmonics distillation, neural vector quantization, and contribution-based pruning.

D.3 LIMITATIONS

Mobile-GS enables fast inference but remains limited by computationally intensive training and dependence on desktop-GPU pretraining. These requirements restrict real-time data acquisition and retraining directly on mobile devices.

  • Training Cost and Complexity: Training Mobile-GS remains computationally intensive despite its fast inference.The added spherical harmonics distillation, neural vector quantization, and neural view-dependent enhancement contribute to this cost.
  • Training Cost and Complexity: Mobile-GS requires pretraining on desktop GPUs before deployment to mobile devices.This requirement limits accessibility for real-time data acquisition and on-device retraining.

E ADDITIONAL QUALITATIVE AND QUANTITATIVE RESULTS · F USRER STUDY

Mobile-GS delivers performance comparable to 3DGS, SortFreeGS, and Speedy-Splat while surpassing them in several scenes and providing superior novel-view rendering quality. A 30-volunteer user study evaluates rendering quality across three public datasets against 3DGS and LocoGS-S.

  • E ADDITIONAL QUALITATIVE AND QUANTITATIVE RESULTS: Mobile-GS achieves performance comparable to 3DGS, SortFreeGS, and Speedy-Splat, surpassing them in several scenes.The comparison uses detailed per-scene quantitative results from Tables 9–11.
  • E ADDITIONAL QUALITATIVE AND QUANTITATIVE RESULTS: Mobile-GS demonstrates superior rendering quality in additional novel-view synthesis results.Figure 8 presents visual comparisons with Speedy-Splat, SortFreeGS, and 3DGS, including highlighted close-ups.
  • E ADDITIONAL QUALITATIVE AND QUANTITATIVE RESULTS: The additional visual comparisons include Ground Truth, 3DGS, Mobile-GS (Ours), and Speedy-Splat.These labels are shown in the comparison materials.
  • E ADDITIONAL QUALITATIVE AND QUANTITATIVE RESULTS: The qualitative and quantitative evaluations cover Tanks&Temples and Deep Blending scenes.These datasets are identified alongside the additional results.
  • F USRER STUDY: The user study compares rendering quality among Mobile-GS, 3DGS, and LocoGS-S.Figure 9 reports preference rates, where a higher rate means more users like the method.
  • F USRER STUDY: 30 volunteers evaluated methods trained and rendered on Mip-NeRF 360, Tank&Temples, and Deep Blending datasets.For fair comparison, 3DGS was also quantized.

G ADDITIONAL MOBILE TESTING

Additional testing on a Snapdragon 8 Gen 3 mobile GPU evaluates cold-start and steady-state rendering performance under thermal throttling, along with power consumption. Mobile-GS maintains 74 steady-state FPS and achieves the lowest measured power consumption among the compared methods.

  • Mobile rendering performance: Evaluations on a Snapdragon 8 Gen 3 GPU report cold-start FPS at launch and steady-state FPS after thermal equilibrium.The measurements analyze performance changes over time and thermal throttling on mobile rendering.
  • Mobile rendering performance: 74 steady-state FPS demonstrates that Mobile-GS supports real-time rendering on mobile devices despite thermal throttling.Mobile FPS can decline over time because of thermal throttling, power limits, GPU clock downscaling, and NPU/CPU frequency limits.
  • Power consumption: Mobile-GS achieves the lowest power consumption among Mobile-GS, 3DGS*, and SortFreeGS* on the Snapdragon 8 Gen 3 mobile GPU.Power draw is measured for Vulkan operators and the MLP using the Qualcomm Trepn Profiler on the Mip-NeRF 360 dataset; 3DGS* and SortFreeGS* are quantized deployment variants.
Loading 2603.11531v1…