CompactSplat: Spatially Adaptive Gaussian Distribution for Feedforward Scene Reconstruction

1 Introduction

• Pixel-wise methods assign one or more Gaussians to every pixel, filling textureless regions with thousands of primitives and increasing representation storage costs.Feed-forward prediction avoids the computationally expensive per-scene optimization used by traditional approaches.Source: §1 · p. 2
• Texture-Aware Partitioner (TAP) and Hierarchical Pyramid Sampler (HPS) guide sparse primitives in homogeneous regions and denser predictions in richly textured areas.TAP partitions images with a quadtree; HPS samples Gaussian attributes from a pooled multiresolution attribute pyramid.Source: §1 · pp. 2–3
• A single partition threshold enables inference-time trade-offs between rendering fidelity and primitive count without retraining or auxiliary modules.The control uses one set of pretrained weights.Source: §1 · pp. 2–3
• CompactSplat achieves competitive quality with fewer primitives on RealEstate10K and ACID, retaining strong standard-metric and high-resolution rendering consistency performance.Cross-dataset evaluation on DTU, DL3DV, and ScanNet reveals quality–compactness trade-offs.Source: §1 · p. 3

2.1 3D Gaussian Splatting

• 3D Gaussian Splatting represents scenes with semi-transparent anisotropic ellipsoids for photorealistic real-time rendering; extensions address stability, anti-aliasing, dynamic scenes, and compression.Examples include Kerbl et al. (2023), training-stability work by Cheng et al. (2024), anti-aliasing by Yan et al. (2024), dynamic-scene methods by Luiten et al. (2023), and compression by Girish et al. (2024).Source: §2.1 · p. 3

2.2 Generalizable 3D Reconstruction

• Feed-forward reconstruction predicts Gaussian attributes from sparse inputs, while alternatives use geometric priors, learned queries, occupied voxels, or entropy-adapted patches.CompactSplat instead uses an image-space quadtree: TAP refines textured regions, and HPS samples pooled attributes and scales primitives by block footprint, enabling one-checkpoint quality–compactness control.Source: §2.2 · p. 3

3.1 Model Architecture

• The dual-path encoder fuses multi-view correspondences and single-view geometric priors, while a multi-scale U-Net supports adaptive Gaussian generation.TAP produces hierarchical masks, and HPS uses them to route feature sampling across decoder levels.Source: Figure 2 · p. 4, §3.1 · p. 4

3.2 Texture-Aware Partitioner (TAP)

• TAP recursively partitions image regions by textural complexity, leaving homogeneous areas in larger blocks and subdividing complex areas into finer blocks.Its threshold controls subdivision and can adjust primitive count at inference without retraining.Source: §3.1 · p. 4, §3.2 · p. 5

3.3 Hierarchical Pyramid Sampler (HPS)

• HPS samples a scale-space attribute field at each partition’s level of detail and modulates Gaussian scale by the partition dimensions.This lets a Gaussian expand to tile broad planar regions while preserving high-frequency detail in complex regions.Source: §3.3 · pp. 5–6

3.4 Training Strategy

• Training first fixes the threshold at τ = 0 for dense geometric initialization, then randomly samples τ from U(0, τmax) during sparsity-aware fine-tuning.The two-stage curriculum exposes one set of weights to varying pruning ratios and trains lower-resolution Gaussians to tile homogeneous regions.Source: §3.4 · p. 6

3.5 Training Loss

• The training objective uses direct photometric supervision to jointly optimize pixel-level fidelity and perceptual quality.The perceptual-loss scaling factor λ is set to 0.05.Source: §3.5 · p. 6

4.1 Main Results

• Evaluation covers RealEstate10K and ACID, with cross-dataset testing on DTU, DL3DV, and ScanNet; metrics include image fidelity, HRRC, and Gaussian-count measures.YonoSplat is evaluated at 224 × 224, whereas other baselines use 256 × 256, which may marginally inflate its metrics.Source: §4.1 · pp. 7–8
• Under magnification, CompactSplat preserves watertight geometry and coherent structures, while baseline renderings can show spatial voids, noisy boundaries, and artifacts.Figure 3 compares RGB and depth renderings at ×1, ×2, and ×4; the caption describes gaps and spurious near-field artifacts in conventional representations.Source: Figure 3 · p. 7, §4.1 · p. 8
• CompactSplat matches or exceeds leading baselines across standard fidelity and HRRC metrics while reducing Gaussian count on RealEstate10K and ACID.Tables 1 and 2 report novel-view synthesis performance on RealEstate10K and ACID, respectively.Source: §4.1 · p. 8, Table 1 · p. 8, Table 2 · p. 8

4.2 Analysis of Dynamic Sparsity Control

• Reducing GPP by adjusting threshold τ yields substantial gains in compactness with only marginal degradation in standard fidelity metrics within a reasonable operating range.The analysis describes a resilient Pareto frontier and a flexible control for reducing memory footprint and rendering overhead while preserving scene integrity.Source: §4.2 · p. 9, Figure 4 · p. 9
• Lower GPP settings use expanded, scale-modulated primitives for smoother planar surfaces, while higher GPP settings allocate denser primitives to recover fine edges and complex boundaries.Figure 5 visualizes this qualitative trade-off across GPP settings.Source: Figure 5 · p. 9

5 Conclusion

• CompactSplat combines a Texture-Aware Partitioner with a Hierarchical Pyramid Sampler to allocate primitives according to regional complexity.The framework departs from rigid pixel-wise prediction.Source: §5 · p. 9
• The scale-aware representation achieves state-of-the-art rendering fidelity across multiple datasets while supporting dynamic sparsity control during inference.The conclusion describes this control as enabling on-the-fly trade-offs between reconstruction quality and memory footprint.Source: §5 · p. 9
• CompactSplat delivers compact 3D priors with flexible trade-offs between reconstruction quality and memory footprint.This is the conclusion’s stated outcome of its content-aware allocation approach.Source: §5 · p. 9

A.1 Preliminaries

• Generalizable 3DGS infers a scene representation from sparse source views, their camera intrinsics, and extrinsics in a single forward pass.The standard model produces Gaussian primitives for every pixel coordinate.Source: §A.1 · p. 14
• The dense prediction paradigm uses vision-backbone features unprojected into 3D with camera parameters and can be supervised end-to-end through differentiable rasterization.The passage cites Ranftl et al. (2021), Zhang et al. (2022), Wang et al. (2024a), Fang et al. (2025), Gao et al. (2025), and Wang et al. (2025) for vision-backbone geometric priors.Source: §A.1 · p. 14
• Rigid pixel-to-Gaussian mapping creates spatial redundancy in homogeneous regions and fails to exploit piecewise-planar surface priors.The paper identifies this as a bottleneck in representational efficiency that CompactSplat seeks to address.Source: §A.1 · p. 14

A.2 Encoder Architecture

• A dual-branch encoder extracts monocular and multi-view features, following DepthSplat and subsequent SurfSplat architectures.The implementation states that the encoder is otherwise unchanged for reproducibility.Source: §A.2 · pp. 14–15
• The multi-view branch uses a stride-2 ResNet-style backbone and a six-layer multi-view Swin Transformer for view aggregation.The Transformer layers use self- and cross-attention and output multi-view-aware features.Source: §A.2 · p. 14
• Plane-sweep stereo warps source-view features to a reference view at uniformly sampled candidate depths and compares them by dot-product similarity.This constructs geometric consistency between views.Source: §A.2 · p. 14
• Monocular features from Depth Anything V2 are upsampled to the cost-volume resolution and combined with multi-view features for depth prediction.The concatenated features are processed by a 2D U-Net; a softmax over depth followed by weighted summation yields the predicted depth map, which is refined hierarchically.Source: §A.2 · pp. 14–15
• Predicted depth reconstructs Gaussian positions, while an additional DPT head predicts other attributes from the image, depth, and encoder features.The listed attributes include scale multipliers, high-frequency spherical-harmonic coefficients, and opacity.Source: §A.2 · p. 15
• The implementation sets downsample scale s to 4 and channel counts C and D to 128; monocular feature channels Cm vary by model size.Cm is 64 for small, 96 for base, and 128 for large; Gaussian processing adopts Surface Continuity Prior and Forced Alpha Blending from SurfSplat.Source: §A.2 · p. 15

• The Texture-Aware Partitioner uses a maximum block size of Smax = 16, and sparsity-aware fine-tuning samples the dynamic partition threshold uniformly up to τmax = 0.02.Source: §A.3 · p. 15
• Training uses AdamW with cosine annealing, a 2 × 10−6 learning rate for the pretrained backbone, and 2 × 10−4 for newly initialized layers.Source: §A.3 · p. 15
• The Depth Anything V2 backbone variants are Ours-S, Ours-B, and Ours-L; they train for 300K or 600K iterations with batch sizes of 16 or 8, respectively.Ours-S uses 300K iterations and batch size 16; Ours-B and Ours-L use 600K iterations and batch size 8.Source: §A.3 · p. 15

• The reported cross-dataset evaluation is described as testing outside CompactSplat’s primary training distribution.Source: §A.4 · p. 15
• CompactSplat achieves rendering fidelity comparable to SurfSplat on cross-dataset evaluations while yielding a more compact, memory-efficient representation.Source: Table 3 · p. 15, §A.4 · p. 15
• Table 4 is identified as a comparison with non-pixel-aligned methods on RE10K.Source: Table 4 · p. 16

A.5 Comparison with other non-pixel-aligned methods

• Training and camera handling differ across the compared methods, including published results for TokenGS and C3G and a two-view re-evaluation for VolSplat.TokenGS and CompactSplat use two 256 × 256 inputs; C3G reports 224 × 224 training inputs and test-time target-camera alignment.Source: §A.5 · p. 16

A.6 Extended Qualitative Results

• Figure 6 shows additional reconstructions across diverse environments, described as photorealistic with watertight geometry and dynamic control over compactness.Source: Figure 6 · p. 16, §A.6 · p. 16

A.7 Limitations and Future Directions.

• The current quadtree uses rigid orthogonal grids and primarily removes within-view redundancy, leaving shape-aware subdivision and cross-view pruning for future work.Source: §A.7 · p. 16