VolSplat: Rethinking Feed-Forward 3D Gaussian Splatting with Voxel-Aligned Prediction

• Feed-forward reconstruction replaces slow per-scene optimization with a single learned inference pass, supporting real-time applications and large-scale reconstruction.Source: §1 · p. 2
• Pixel-aligned methods suffer from calibration and discretization errors, inconsistent cross-view sampling, and density distributions rigidly tied to image pixels.Source: §1 · p. 2
• VolSplat aggregates multi-view features in a shared 3D voxel grid, then refines the grid and predicts Gaussian primitives directly from voxels.Source: §1 · pp. 2–3
• Voxel alignment reduces view-dependent inconsistency and alignment-induced errors by fusing information volumetrically before Gaussian prediction.Source: §1 · pp. 2–3
• VolSplat achieves state-of-the-art performance on RealEstate10K, ScanNet, and ACID, outperforming comparable pixel-aligned feed-forward models.Source: §1 · pp. 2–3

Novel view synthesis

• Novel-view synthesis methods use explicit geometry, view interpolation, or dense ray sampling, each requiring accurate geometry, dense viewpoints, or both.Source: §2 · p. 3

3D voxelization

• Voxel methods evolved from memory-intensive dense grids toward sparse structures and now commonly support 3D CNNs or organize advanced rendering representations.Source: §2 · p. 3

Feed-forward 3D Gaussian Splatting

• Feed-forward 3DGS predicts Gaussians in one pass; pixelSplat, MVSplat, and DepthSplat respectively emphasize epipolar processing, cost-volume fusion, and monocular depth features.Source: §2 · pp. 3–4

VolSplat pipeline

• VolSplat extracts image features, builds cost volumes, predicts depths, unprojects features into voxels, refines them sparsely in 3D, and renders from voxel-predicted Gaussians.Source: Figure 2 · p. 4

3.1 Preliminary and Observation

• Pixel-aligned prediction fixes the Gaussian count at H × W × N, causing over-densification on textureless surfaces and insufficient representation for complex geometry.Its Gaussian positions are determined from predicted depth and pixel locations, tying the 3D representation to the 2D image grid.Source: §3.1 · pp. 4–5

3.2 3D Feature Construction

• The method extracts cross-view features, builds plane-sweep cost volumes, and regresses dense per-pixel depths for lifting image features into a shared 3D space.Cross-view attention exchanges information between neighboring views, while cost-volume matching supplies the geometric prior for 3D construction.Source: §3.2 · p. 5
• VolSplat voxelizes lifted points by voxel size and average-pools their associated image features into voxel features for volumetric reasoning.Each point is assigned an integer voxel index, and features from points within a voxel are aggregated along the channel dimension.Source: §3.2 · pp. 5–6

3.3 Feature Refinement and 3D Gaussians Prediction

• A sparse 3D U-Net predicts residual voxel features that are added to the original grid, enabling multi-scale geometric refinement while preserving coarse information.Hierarchical sparse convolutions, encoder-decoder stages, skip connections, and residual updates support efficient local and global context fusion.Source: §3.3 · p. 6
• The refined voxel grid predicts per-voxel Gaussian centers, opacities, covariances, and colors, which are trained end-to-end with photometric and perceptual losses.Gaussian centers are constrained to localized neighborhoods around voxel centroids, with the refinement scale set to three times the voxel size.Source: §3.4 · pp. 6–7, §3.3 · p. 6

• VolSplat compares favorably across 6, 12, and 24 input views, consistently achieving the best performance across all metrics on RealEstate10K.The comparison includes pixel-aligned and post-processing baselines retrained under the same evaluation setup.Source: Table 1 · p. 7

4.2 Experimental Results and Analysis

• VolSplat achieves state-of-the-art results on both RealEstate10K and ScanNet, while pixel-aligned models degrade substantially under sparse multi-view inputs.The comparison covers mainstream pixel-aligned models and variants; VolSplat is also reported to outperform all baselines on ScanNet.Source: §4.2 · p. 8, Table 2 · p. 10
• VolSplat reduces floaters and artifacts, producing cleaner object boundaries and more coherent reconstructions on RealEstate10K and ScanNet.The qualitative comparisons attribute the improvement to resolving multi-view alignment issues in the 3D feature representation.Source: Figure 4 · p. 8, §4.2 · p. 9, Figure 5 · p. 9
• VolSplat maintains significantly higher performance than baselines when transferred from RealEstate10K to ACID without fine-tuning.This evaluates zero-shot transfer from indoor training data to outdoor scenes.Source: §4.2 · pp. 9–10, Table 3 · p. 10
• VolSplat allocates more Gaussians to geometrically complex regions and fewer to flat or empty areas, unlike the constant-density behavior of pixel-aligned methods.The density analysis describes this as a more efficient, geometry-aware representation.Source: §4.2 · pp. 10–11, Figure 7 · p. 10

4.3 Ablation Study

• A voxel size of 0.1 provides the best quality-efficiency trade-off, whereas overly small or large voxels lose coherent context or fine detail.The voxel-size study reports degraded performance at 0.05 and with larger voxels.Source: Table 4 · p. 11, §4.3 · p. 11
• Removing or replacing the sparse 3D refinement decoder reduces performance, supporting the proposed residual 3D U-Net design.The ablation compares removing refinement, removing residual prediction, and replacing the 3D U-Net with a sparse 3D CNN.Source: Table 5 · p. 11, §4.3 · pp. 11–12

5 Conclusion

• VolSplat shifts feed-forward Gaussian prediction from 2D pixels to a unified 3D voxel-aligned representation.The framework constructs voxel features and predicts Gaussians directly from them.Source: §5 · p. 12
• Voxel alignment decouples the reconstructed scene from input-view constraints while enabling adaptive Gaussian density and resolving alignment ambiguities.The stated outcome is more geometrically consistent and faithful reconstruction for downstream tasks.Source: §5 · p. 12

A More Implementation Details

• A weight-sharing ResNet extracts multi-scale features, followed by local-window cross-view attention to aggregate neighboring-view information.The architecture then proceeds to multi-view depth processing.Source: A More Implementation Details · p. 18
• Training uses AdamW with separate learning-rate groups, six input views, eight target views, 256 × 256 resolution, and RealEstate10K-to-ScanNet fine-tuning.ACID is evaluated zero-shot, and the anchor-gap range expands progressively during early iterations.Source: A More Implementation Details · p. 18
• Evaluation selects target novel views from disjoint camera positions and assesses eight target views per scene under fixed frame-gap rules.This protocol is designed for controlled comparison with prior feed-forward 3DGS methods.Source: A More Implementation Details · p. 18
• The authors state that the complete VolSplat codebase will be open-sourced.Source: A More Implementation Details · p. 18

• VolSplat tops all image-quality metrics while retaining competitive inference efficiency against competing approaches.The comparison evaluates all methods with 6 input views on RealEstate10K using a single NVIDIA H20 GPU.Source: B More Experimental Analysis · pp. 18–19
• The reported relative rankings reflect a controlled and fair comparison despite hardware-dependent differences in absolute runtimes.Source: B More Experimental Analysis · p. 19

• VolSplat assumes static scenes, so moving objects or changing environments can produce ghosting or blurring in rendered novel views.The limitation arises because multi-view feature aggregation and voxel construction rely on geometric consistency across views.Source: C Limitation and Societal Impacts · p. 19
• High-fidelity reconstructions from limited data could be misused for deepfakes or unauthorized digital replicas of private spaces.The paper recommends watermarking or authentication protocols for sensitive deployments.Source: C Limitation and Societal Impacts · p. 19

• Additional RealEstate10K comparisons include VolSplat, ground truth, and five baselines under 6, 12, and 24 input views.The displayed baselines are AnySplat, WorldMirror, MVSplat, DepthSplat, and GGN.Source: D More Visual Comparisons · pp. 19–21, Figure C · p. 22
• The visual comparisons are organized to examine performance as the number of input views increases from 6 to 24.Source: D More Visual Comparisons · pp. 19–21, Figure C · p. 22