PaperlayerView exact revision on arXiv

Source-linked AI summary

Mobile-4DGS: Unified Static-Dynamic Real-time Mobile Gaussian Splatting

Xiaobiao Du, Beixi Hao, Zhen Fang, Tianqing Zhu, Richard Hartley, Xin Yu

arXiv:2610.05289v1cs.CV

TL;DR

Mobile-4DGS targets the storage, redundancy, and per-frame computation that hinder mobile deployment of static and dynamic Gaussian Splatting. It combines compact appearance modeling, multi-view density control, explicit temporal representation, and selective depth-order reuse; experiments report competitive visual quality alongside reduced resource use and strong mobile rendering performance.

  • Problem

    Deploying static and dynamic Gaussian Splatting on resource-constrained mobile devices is hindered by SH storage, redundant primitives, deformation inference, and repeated depth ordering.

    Source: §I · p. 1

  • Method

    Mobile-4DGS combines compact SH appearance modeling and multi-view densification with an explicit continuous-time static-dynamic representation and a depth-order certificate for playback.

    Source: §I · pp. 2–3

  • Results

    On N3DV on iPhone 14, Mobile-4DGS has the highest rendering throughput, smallest storage footprint, and shortest training time while maintaining competitive SSIM.

    Source: B. Qualitative and Quantitative Results · p. 13

  • Takeaways & Limitations

    The results demonstrate mobile continuous-time dynamic rendering with a model of only a few megabytes, while static scenes retain competitive quality and high rendering speed.

    Source: §VI · p. 16

Abstract

from arXiv · show

Recent advances in 3D Gaussian Splatting (3DGS) have achieved remarkable performance in novel view synthesis, yet deploying both static and dynamic Gaussian representations on resource-constrained mobile devices remains challenging due to heavy storage, redundant primitives, and costly per-frame computation. We present Mobile-4DGS, a unified lightweight framework for high-fidelity real-time static and dynamic Gaussian rendering on mobile platforms. For compact appearance modeling, we introduce a Monte Carlo Specular Energy Aggregator that compresses high-order radiance residuals into the first-order Spherical Harmonics (SH), together with an Attribute-Conditioned SH Enhancement module whose predicted offsets are pre-baked before inference. We further propose a Multi-View Alpha-Based Densification and Pruning strategy to suppress redundant primitives while maintaining multi-view consistency. For dynamic scenes, we develop a compact explicit 4D representation by constructing second-order Gaussian motion, learnable temporal support, and a binary static-dynamic partition, enabling continuous-time modeling without runtime deformation networks. Based on this partition, a Depth-Order Certificate selectively reuses previously committed depth orders to reduce re-projection, sorting, merging, and index-buffer updates during playback. Extensive experiments on static and dynamic scenes demonstrate that Mobile-4DGS substantially reduces storage and rendering overhead while maintaining competitive visual quality, enabling real-time 3D and 4D Gaussian Splatting on mobile devices. \textcolor{magenta}{\href{https://xiaobiaodu.github.io/mobile-4dgs-project/}{Code has been released: https://xiaobiaodu.github.io/mobile-4dgs-project/}}.

I. INTRODUCTION

Mobile-4DGS addresses the storage and per-frame computation burdens that make static and dynamic Gaussian Splatting difficult to deploy on mobile devices. It combines compact appearance and density control with an explicit static-dynamic representation and selective depth-order reuse.

I. INTRODUCTION

  • Mobile-4DGS matches 3DGS [1] and Mobile-GS [2] in rendering quality while using fewer primitives and achieving higher FPS on Snapdragon 8 Gen 3.The framework also uses WebGL for cross-platform static and dynamic novel-view rendering.Source: Figure 1 · p. 1
  • Mobile deployment is constrained by third-order SH storage, redundant densification, per-frame deformation inference, and repeated depth ordering under motion.Third-order SH requires 48 floating-point parameters per Gaussian, while motion invalidates cached depth orders and triggers repeated projection, sorting, and draw-list updates.Source: §I · pp. 1–2
  • Mobile-4DGS unifies static and dynamic rendering in a compact continuous-time representation, with static scenes as the zero-motion case and eight additional temporal scalars per dynamic Gaussian.Both modes share the appearance, compression, density-control, and rendering pipeline.Source: §I · pp. 2–3
  • Monte Carlo aggregation compresses high-order specular energy into a lower-order SH representation, while baked attribute-conditioned offsets restore fine detail without inference-time overhead.The representation separates a view-independent base from directional components, reducing SH memory overhead without costly offline distillation or pre-training.Source: §I · pp. 2–3
  • Multi-view alpha-weighted error guides densification and pruning to control redundant Gaussians while maintaining structural consistency across views.The strategy uses stratified camera sampling and focuses densification on regions with high multi-view reconstruction error.Source: §I · pp. 2–3
  • A depth-order certificate reuses a committed draw list when a closed-form travel bound confirms validity, skipping re-projection, sorting, merging, and index upload.The binary learned partition lets the renderer cache the static subset's depth order per camera.Source: §I · pp. 2–3

II. RELATED WORK

Related work covers neural and Gaussian scene representations, along with methods for managing Gaussian count, compression, and temporal variation. The preliminary material defines the Gaussian parameters, rendering order, alpha blending, and SH basis used in the paper.

II. RELATED WORK

  • 3DGS uses anisotropic Gaussians and tile-based rasterization; later work addresses count, level of detail, kernels, optimization, densification, and sorting.Examples include Scaffold-GS, Octree-GS [42], Taming 3DGS, MVGS [50], and StopThePop.Source: §II · p. 3
  • Pruning methods use transmittance, Hessian sensitivity, or gradient-opacity scores, while Mobile-GS uses opacity and scale but can lose fine detail.The paper contrasts these approaches with its multi-view alpha-based densification and pruning strategy.Source: §II · p. 3
  • Gaussian compression spans vector quantization, entropy coding, structured representations, contextual models, and neural fields, with trade-offs in training time, index storage, or rendering latency.Mobile-GS distills SH from a high-order teacher; Mobile-4DGS instead directly compresses third-order SH energy into a low-order representation.Source: §II · p. 3
  • Dynamic Gaussian methods model temporal variation with time coordinates, deformation networks, explicit grids, or motion trajectories, but temporal coefficients and network inference can burden mobile rendering.The paper positions its temporal feature aggregation as an alternative intended to retain dynamic fidelity and mobile efficiency.Source: §II · p. 4

III. PRELIMINARY

  • A 3D Gaussian has position, covariance-derived scale and rotation, opacity, and SH color; rendering projects Gaussians, sorts them near-to-far, then alpha-blends them.The covariance describes the anisotropic Gaussian's spatial extent and orientation.Source: §III · p. 4
  • Spherical harmonics expand directional color using basis functions and coefficients, with finite degree controlling coefficient count and higher orders capturing high-frequency view-dependent effects.The preliminary section notes that low-order SH are often preferred for compactness and numerical stability.Source: §III · p. 4

IV. METHODOLOGY

Mobile-4DGS compresses high-order radiance into first-order SH while preserving view-dependent specular information. Attribute-conditioned refinement is baked into the representation before inference.

IV. METHODOLOGY

  • Mobile-4DGS uses first-order SH with 4 coefficients per channel and compresses high-order radiance energy into this compact representation.The method describes this as a substantial reduction in SH storage compared with third-order baselines.Source: §IV · p. 4

A. Monte Carlo Specular Energy Aggregator

  • Monte Carlo directional moments preserve the magnitude and direction of discarded specular residuals in a compact representation.The method extracts first-order directional moments rather than aggregating absolute residuals, which would discard angular direction.Source: A. Monte Carlo Specular Energy… · p. 5
  • The aggregator stores directional and base-color latents, then maps them to first-order SH coefficients using a neural network conditioned on normalized Gaussian position.The resulting coefficients are decoded once before inference, with no per-frame decoding overhead.Source: A. Monte Carlo Specular Energy… · p. 6
  • An Attribute-Conditioned SH Enhancement module predicts residual offsets from Gaussian attributes and pre-bakes them into explicit Gaussian parameters.The residual module uses intrinsic Gaussian attributes and is independent of camera viewing direction.Source: A. Monte Carlo Specular Energy… · p. 6
  • Training begins with third-order SH for 3k iterations, then compresses to first-order SH and continues refinement with the enhancement module.Compression occurs once; subsequent updates maintain the lightweight first-order parameters.Source: A. Monte Carlo Specular Energy… · p. 6

B. Multi-view Alpha-based Densification and Pruning

Mobile-4DGS uses stratified camera sampling and multi-view alpha-weighted error scores to guide densification and pruning. Its criteria aim to preserve multi-view consistency while controlling redundant Gaussian primitives.

B. Multi-view Alpha-based Densification and Pruning

  • The strategy combines multi-view loss-driven evaluation with alpha-based Gaussian scoring to densify poorly reconstructed regions and moderately prune well-reconstructed areas.It is designed to constrain Gaussian count based on multi-view evaluation and maintain multi-view consistency.Source: B. Multi-view Alpha-based… · p. 6, Figure 4 · p. 7
  • Stratified camera sampling selects representative views across angular bins, then shuffles and truncates candidates to meet a computational budget.The scene center is estimated from camera rays, and the camera set is selected for broad angular coverage.Source: B. Multi-view Alpha-based… · p. 6
  • Multi-view alpha-weighted error accumulation identifies Gaussians contributing to high- or low-error regions across sampled views.The aggregated scores help distinguish important Gaussians from redundant ones and avoid overfitting to single-view occlusions or transient artifacts.Source: B. Multi-view Alpha-based… · p. 7
  • Cloning and splitting require both standard gradient criteria and an importance-quantile filter, limiting densification to Gaussians consistently contributing to high-error regions.The filtered masks are intended to improve fitting while controlling primitive count.Source: B. Multi-view Alpha-based… · p. 7
  • Pruning targets Gaussians below the opacity threshold that exceed the pruning-score quantile, culling artifacts while preserving essential scene geometry.This is presented as moderate pruning that maintains overall reconstruction fidelity.Source: B. Multi-view Alpha-based… · p. 7

C. Dynamic Modeling

Mobile-4DGS models dynamic scenes with explicit second-order Gaussian motion, learnable temporal support, and a binary static–dynamic partition. The partition supports continuous-time trajectories while leaving static Gaussians unchanged and available for caching.

C. Dynamic Modeling

  • Each Gaussian stores explicit velocity, acceleration, canonical time, and duration; a binary gate selects a continuous second-order trajectory or exactly zero displacement.Appearance and covariance remain constant over time.Source: C. Dynamic Modeling · pp. 7–8
  • A learnable temporal window modulates dynamic Gaussian opacity so primitives can smoothly appear and disappear around their canonical time.The window helps model visibility changes, occlusions, and topology changes.Source: C. Dynamic Modeling · p. 8
  • The learned binary partition distinguishes static from dynamic Gaussians, with motion evidence promoting misplaced static assignments and sparsity regularization discouraging unnecessary dynamic assignments.The committed partition is frozen at iteration 25k; promotion is limited to at most 1% of Gaussians per step.Source: C. Dynamic Modeling · pp. 8–9
  • Temporal regularization constrains acceleration, duration, and motion extent to address unstable trajectories, broad temporal support, and motion trails.The losses affect only dynamic Gaussians and do not alter static canonical geometry.Source: C. Dynamic Modeling · p. 9
  • Temporal-aware density control samples cameras across non-empty time bins, while motion-state compression stores temporal codebook indices only for dynamic Gaussians.Static Gaussians share an index decoding to an exact zero-motion state, and their unchanged subset can be cached for rendering.Source: C. Dynamic Modeling · pp. 9–10

D. Mobile Rendering of the Gated Representation

The renderer sorts static and dynamic Gaussian streams separately, merges them, and uses a motion-bound certificate to decide when a committed draw order can be reused. Certified reuse avoids repeated sorting and related frame updates, while a relaxed mode permits bounded depth motion.

D. Mobile Rendering of the Gated Representation

  • For a fixed camera, the static stream can be cached; independently sorting static and dynamic streams and merging them yields the same 16-bit quantized order as global sorting.A change in camera depth mapping invalidates the static cache and committed global draw list.Source: D. Mobile Rendering of the Gated… · p. 10
  • A common bound on dynamic depth-key motion certifies reuse when twice the bound is smaller than the smallest relevant neighboring key gap.Static–static pairs are excluded because their relative order cannot change for a fixed camera.Source: D. Mobile Rendering of the Gated… · pp. 10–11
  • The optional relaxed policy accepts motion up to a specified budget in mean Gaussian radii, but may allow small local ordering changes.The default mobile-viewer setting is 0.5 mean radii; the zero-budget case gives the exact certificate.Source: D. Mobile Rendering of the Gated… · p. 11
  • Reused frames keep the committed draw list and GPU index buffer, reducing communication from 4N bytes for a newly committed list to O(1).The worker skips primitive re-projection, counting sort, and stream merging on reuse.Source: D. Mobile Rendering of the Gated… · p. 11

E. Implementation

The implementation uses a WebGL-based framework to render Mobile-4DGS across platforms in browsers.

E. Implementation

  • The framework uses WebGL to render directly in browsers without external dependencies.Source: E. Implementation · p. 11
  • To handle non-commutative alpha compositing, depth sorting runs asynchronously in a WebWorker while the CPU updates visibility order.Source: E. Implementation · p. 11
  • The rendering loop can exceed the 120 FPS display limit by decoupling depth sorting and visibility-order updates.Source: E. Implementation · p. 11

A. Implementation Details

Implementation details cover the training schedule, mobile evaluation hardware and protocol, and the datasets used in reported evaluations.

A. Implementation Details

  • Mobile-4DGS is trained for 30k iterations without pretraining or distillation, beginning with third-order SH for 3k iterations before conversion to first-order SH.The Monte Carlo aggregator samples 2048 uniform points on the unit sphere.Source: A. Implementation Details · pp. 11–12
  • The reported evaluation scopes include Mip-NeRF 360 on Snapdragon 8 Gen 3, Tanks and Temples and Deep Blending, and N3DV FPS on iPhone 14.Mip-NeRF 360 scenes are categorized as indoor and outdoor; the N3DV table reports smartphone FPS.Source: Table II · p. 12, Table III · p. 12, Table IV · p. 12
  • Static mobile rendering is evaluated on a Snapdragon 8 Gen 3 GPU and dynamic rendering on an iPhone 14/A15 using offscreen frame-rate measurements.The protocol averages consecutive runs after warm-up to mitigate thermal and initialization effects.Source: A. Implementation Details · p. 12

B. Qualitative and Quantitative Results

The evaluations report that Mobile-4DGS balances competitive visual quality with reduced storage and faster rendering in static and dynamic mobile settings. Qualitative comparisons highlight preserved detail in static scenes and faithful rendering of challenging dynamic content.

Static Evaluation and Mobile Performance

  • Mobile-4DGS maintains competitive image-quality metrics while delivering fast FPS and low memory footprint across tested indoor and outdoor environments.The comparison reports low storage costs and faster convergence than prior methods, including Speedy-Splat and Mobile-GS.Source: B. Qualitative and Quantitative Results · p. 13, Figure 6 · p. 13

Dynamic Assessment and Mobile Results

  • On N3DV on an iPhone 14, Mobile-4DGS provides the highest rendering throughput, smallest storage footprint, and shortest training time while maintaining competitive SSIM.It has comparable Gaussian counts to OMG4 and improves rendering and training efficiency over that method.Source: B. Qualitative and Quantitative Results · p. 13, Figure 7 · p. 14

Qualitative Results

  • In dynamic comparisons, Mobile-4DGS renders rapid motion and object interactions with sharper boundaries and fewer motion artifacts than competing methods.It uses substantially fewer primitives and less storage than Real-Time4DGS, and a more compact representation than OMG4.Source: B. Qualitative and Quantitative Results · p. 14

C. Ablation Study

Ablations show that the appearance components and multi-view densification and pruning jointly support the quality-efficiency trade-off, while the Depth-Order Certificate offers controllable ordering reuse for dynamic rendering.

Component Ablations

  • The full model achieves the best quality-efficiency trade-off, while removing MC-SEA or the SH offset degrades reconstruction quality.The ablation table covers MC-SEA, the attribute-conditioned SH offset, and multi-view densification and pruning.Source: C. Ablation Study · pp. 14–15, Table V · p. 15
  • Replacing multi-view densification with single-view densification increases Gaussian count, memory use, and storage cost while reducing FPS; removing pruning also increases model size and reduces speed.These ablations support the role of the multi-view strategy in limiting redundant primitives.Source: C. Ablation Study · pp. 14–15

Spherical Harmonic Decomposition

  • The 0th-order SH captures base diffuse color and global illumination, while 1st-order and Δ1st-order components capture high-frequency structure and refine local context.The reported decomposition also shows the first-order strategy capturing intricate textures and complex specular appearance.Source: C. Ablation Study · p. 15, Figure 8 · p. 15

Depth-Order Certificate Ablation

  • The Depth-Order Certificate trades bounded depth-order deviation for higher reuse and reduced worker computation and index-buffer communication; r = 0 preserves strict order.Higher reuse is observed with denser frame requests, indicating benefit from stronger inter-frame temporal coherence.Source: C. Ablation Study · p. 15, Table VI · p. 15

D. Monte Carlo K Sampling Point Analysis

The sampling analysis finds that increasing Monte Carlo sampling density improves reconstruction quality up to a saturation region, with K = 2048 selected as the default to limit computation.

Sampling Density

E. Camera Count in Multi-view Alpha-based Densification

The camera-count analysis finds that more input views improve PSNR and reduce Gaussian count, leading the authors to use six views. The conclusion presents Mobile-4DGS as a unified explicit representation offering a practical trade-off across visual fidelity, model size, training cost, and real-device throughput.

E. Camera Count in Multi-view Alpha-based Densification

  • Increasing camera count from 2 to 6 significantly improves PSNR and reduces Gaussian count, so the authors sample 6 views for a trade-off between rendering performance and training burden.The reported reduction in primitives is attributed to stronger multi-view constraints helping localize geometry for densification and avoid overfitting and redundancy.Source: E. Camera Count in Multi-view… · p. 16

VI. CONCLUSION

  • Mobile-4DGS offers a favorable trade-off among visual fidelity, model size, training cost, and real-device throughput rather than aiming to dominate every unconstrained reconstruction metric.The conclusion reports continuous-time dynamic rendering on mobile hardware with a model of only a few megabytes, while static scenes retain competitive quality and high rendering speed.Source: §VI · p. 16

Related papers

Loading 2610.05289v1…