Is Gaussian Splatting Becoming Neural Again? A Taxonomy and Controlled Study of Learned Parameterization

1. Introduction

• 3DGS directly optimizes anisotropic primitives and rasterizes their projected footprints, whereas neural systems can use networks to generate or share Gaussian parameters.In generalizable models, splats may serve as an output format rather than the sole information-bearing representation.Source: §1 · p. 1
• A binary neural label conflates components with different pipeline roles and prevents meaningful comparison of their design choices.The paper asks which parameters are stored or predicted, what limitations each neuralization addresses, and what is gained or lost by moving information into shared parameters.Source: §1 · p. 1
• The paper contributes a five-axis taxonomy, an analysis of 19 representative methods, and a controlled study finding benefits from shared attribute decoding but not additional structure decoding.Spatially shared attribute decoding improves all three image metrics in the study.Source: §1 · p. 1

2. From Fields to Splats and Back to Functions

• Neural radiance fields map location and direction to density and color, rendering rays through numerical integration; later systems accelerate or restructure field representations.The passage notes that “neural” and “implicit” are not synonyms: Plenoxels is an explicit field without a neural network, while Point-NeRF is point-based yet neural.Source: §2 · pp. 1–2
• Vanilla 3DGS stores independent trainable position, rotation, scale, opacity, and spherical-harmonic coefficients for each primitive, then rasterizes projected Gaussians.Densification and pruning change topology during training.Source: §2 · p. 2
• Neuralized explicit representations generate Gaussian state from shared parameters while retaining explicit splats at render time.The shared structure may be an anchor grid, hash table, tri-plane, or continuous field, and a decoder may predict appearance, shape, or existence.Source: §2 · p. 2
• NeRF and 3DGS can complement one another by sharing continuous spatial information with a Gaussian branch.The cited work also describes learned radiance representations transported across architectures, including NeRF/3DGS conversion.Source: §2 · p. 2

3.1. A five-axis audit vector

• The audit vector records five architectural tests, while its equal-weight score is only a broad ordering and summary because methods with the same score can use neural components differently.The vector is the main description; the score does not imply that the axes matter equally in every application.Source: §3.1 · pp. 2–3
• Attribute decoding is neural when shared learned functions produce one or more render-time attributes; partial decoding leaves substantial attributes independent.The audit distinguishes direct per-primitive values from shared-function outputs.Source: §3.1 · p. 2
• Spatial sharing requires multiple splats to query a common spatially indexed latent structure; a global image feature alone does not qualify.Examples include anchor features, hash grids, voxel fields, tri-planes, and MLP fields.Source: §3.1 · p. 2
• Learned view conditioning maps view information to Gaussian attributes, while topology generation predicts Gaussian centers, offsets, activation, or existence.Fixed spherical harmonics and heuristic split/prune rules alone do not satisfy these tests; learned offsets around directly optimized parents count as partial topology generation.Source: §3.1 · p. 2
• Amortized inference uses shared weights to infer a Gaussian scene from test-time observations without full per-scene reconstruction optimization.This separates generalizable reconstruction from scene-specific neural parameterization.Source: §3.1 · p. 2

3.2. What is not counted as neuralization

• Neural supervision, initialization, or training-view generation does not by itself count as neuralization when it does not change deployed scene storage or inference.Pruning, quantization, and entropy coding likewise do not increase the audit score unless a learned function decodes Gaussian attributes.Source: §3.2 · p. 3

4. Taxonomy: Problem →Sharing Mechanism

• Anchor sharing addresses duplicated appearance and geometry among nearby Gaussians by trading primitive independence for compact, correlated state.Scaffold-GS uses anchors and shared features, Hash-GS adds multiresolution spatial encoding, and SOGS enriches reduced anchor features; codebooks and entropy coding address the same symptom without neuralization.Source: §4 · p. 3
• Learned appearance functions offer an alternative to finite-order spherical harmonics, whose basis capacity is fixed, while direct coefficients remain compressible.The cited contrast includes view-conditioned color and opacity prediction and non-decoder regularization and truncation of spherical harmonics.Source: §4 · p. 3
• For sparse observations, methods address inaccurate initialization and local overfitting through depth, propagation, density control, or a neural spatial prior.The passage frames the design question as whether shared structure reduces otherwise independent degrees of freedom.Source: §4 · p. 3
• For dynamic scenes, learned temporal deformation functions provide correspondence beyond per-frame copies, which scale poorly.The cited splat approaches bring temporal functions to Gaussian representations while retaining rasterization.Source: §4 · p. 3
• Amortized inference marks cross-scene generalization: shared weights infer Gaussian scenes from observations, with splats serving as renderer-facing outputs.Examples include feed-forward Gaussian prediction, pixel-to-splat mapping, and depth estimation coupled with splatting.Source: §4 · p. 3
• For semantics and generation, splats provide a differentiable, fast 3D interface while information for interaction can reside in a shared latent space.The renderer remains explicit.Source: §4 · p. 3

5.1. Representative architectures

• Table 1 covers 19 representative methods, from direct optimization to feed-forward reconstruction, and excludes methods using 3DGS only as an unrelated downstream component.Source: §5.1 · p. 3
• VDGS and NeRF-GS both have N = 0.4, but VDGS neuralizes view-dependent appearance while NeRF-GS introduces shared continuous spatial features.Source: §5.1 · p. 3
• Among 14 methods with nonzero scores, all decode attributes neurally; 13 predict some structure, 10 share spatial information, six amortize reconstruction, and four use learned view conditioning.Source: §5.1 · p. 3
• Feed-forward systems make a larger conceptual shift: a network predicts the Gaussian scene directly, replacing per-scene optimization.Source: §5.1 · pp. 3–4

5.2. Comparison with published architectures

• 19.40 PSNR for 3DGS, 20.70 for 2DGS, and 21.07 for SplatFields [8] are reported under the three-view DTU protocol; the comparison also changes primitive formulation and training procedure.Source: §5.2 · pp. 4–5
• NeRF-GS [38] performs best in the common mip-NeRF 360 comparison, but image quality does not increase monotonically with neuralization score N.The table reports results under a common protocol; the score is an architectural descriptor, not a performance predictor.Source: §5.2 · p. 4, Table 2 · p. 5

6. Controlled Neuralization Study

• Four variants share the same rasterizer, initialization, densification, views, loss, iterations, and spherical-harmonic configuration, varying only the source of Gaussian attributes.E0 directly optimizes attributes; E1 adds neural appearance, E2 adds shared spatial features and opacity decoding, and E3 also decodes covariance and position offsets.Source: §6 · p. 5

6.1. Results

• E1 improves PSNR by 0.27 dB and reduces LPIPS from .214 to .207; E2 adds 0.28 dB, reaching 27.76 dB PSNR, .820 SSIM, and .196 LPIPS.Source: §6.1 · p. 5
• 0.55 dB PSNR, .005 SSIM, and .018 LPIPS: E2 improves all three metrics over direct 3DGS.Table 3 reports the controlled mip-NeRF 360 comparison.Source: Table 3 · p. 5, §6.1 · p. 5
• Compared with E2, E3 decreases PSNR by 0.03 dB and SSIM by .006 and increases LPIPS by .008; direct geometric parameters remain valuable.The study attributes the E3 trade-off to reduced local flexibility when structure is forced through a shared decoder.Source: §6.1 · p. 5

• Shared fields or anchor decoders can reduce storage and couple gradients when neighboring splats have correlated values, especially under sparse evidence or at scale.Geometry, pruning, and stochastic topology can provide related benefits without neuralization.Source: §7 · p. 5
• Feed-forward models enable rapid inference on unseen inputs, but depend on the training distribution and require a larger global model.Per-scene direct optimization instead creates one parameter set per scene.Source: §7 · p. 5
• View-conditioned decoders can exceed fixed spherical-harmonic bases, while shared parameters can make local editing harder by affecting many splats.Evaluation should distinguish render-time primitive access from the locality of the parameters generating them.Source: §7 · p. 5
• Decoded attributes may require feature queries and MLP evaluation before splatting, unlike vanilla splats that serialize and rasterize without a neural runtime.Efficiency reporting should separate decoder latency, materialization cost, and rasterizer FPS.Source: §7 · p. 6

• Neuralized 3DGS predicts or materializes finite primitives for alpha compositing, unlike NeRF's repeated volumetric queries along camera rays.Source: §8 · p. 6
• The described progression is neural field → direct splats → neuralized explicit splats.Source: §8 · p. 6
• The design question is where sharing helps and which attributes should remain explicit; the controlled study finds appearance and opacity sharing beneficial, but no further gain from decoding geometry.Source: §8 · p. 6

• The literature set covers representative architectures, not every published 3DGS variant, so its counts describe only the selected methods.Source: §9 · p. 6
• Equal weights in the taxonomy's summary equation are a simple summary, not a claim that every axis matters equally for every application.Source: §9 · p. 6
• The controlled experiment measures novel-view image quality on mip-NeRF 360, not memory, end-to-end latency, editing locality, or cross-scene generalization.Source: §9 · p. 6
• E1–E3 represent common neuralization forms but are not exact reproductions of every anchor-, hash-, or tri-plane-based method.Source: §9 · p. 6

• 3DGS combines neural parameter sharing with explicit splat rendering rather than returning to NeRF.Source: §10 · p. 6
• The taxonomy separates five neuralization pathways and clarifies the roles of anchor decoders, spatial fields, deformation networks, and feed-forward predictors.Source: §10 · p. 6
• Shared appearance and opacity improve all evaluated metrics, whereas additional structure decoding slightly reduces quality.Source: §10 · p. 6