DensiTok: Making Feed-Forward 3D Gaussian Splatting See More Views Than It Is Given

1 INTRODUCTION

• With few unposed views, geometry tokens cover only observed surfaces, leaving holes, floaters, and blurred novel views.The missing evidence is upstream of the reconstruction heads: unobserved regions have no tokens.Source: §1 · p. 2
• Image-space remedies synthesize extra views and re-encode them, adding generation and encoder costs without enforcing cross-view consistency.The resulting pixels are compressed into tokens, although the reconstruction heads need only those tokens.Source: §1 · p. 2
• DensiTok augments unobserved-view token sets between the geometry encoder and reconstruction heads, with the encoder frozen and heads frozen or minimally adapted.It compresses the encoder’s layer stack into a compact latent per view, then expands completed latents into tokens for every layer.Source: §1 · p. 2
• LAM-VAE adaptively fuses feature levels into compact per-view latents, which a camera-conditioned DiT completes for interpolated views in one flow-matching step.New camera locations are interpolated along the path through the backbone’s predicted cameras for observed views.Source: §1 · pp. 2–3
• Across RealEstate10K and DL3DV, DensiTok improves sparse-view rendering quality across three feed-forward 3DGS backbones.Where it reaches dense-view quality, it does so in less time than dense-view reconstruction.Source: §1 · p. 3

2.1 FEED-FORWARD GAUSSIAN SPLATTING

• Sparse unposed inputs limit feed-forward Gaussian-splatting models to visible scene parts, motivating DensiTok’s completion of tokens for additional viewpoints.General backbones accept arbitrary numbers of views, but limited overlap leaves pose and depth ambiguous.Source: §2.1 · p. 3

2.2 GENERATIVE PRIORS FOR 3D RECONSTRUCTION

• Image-space generative priors synthesize views before reconstruction or refine rendered sequences, but add generation and reconstruction costs and can introduce cross-view geometric artifacts.DensiTok instead generates in the backbone’s token space and uses the same heads to decode one Gaussian scene.Source: §2.2 · p. 3

2.3 GENERATION IN 3D-AWARE LATENT SPACES

• Unlike GLD, which generates full-width feature levels in cascaded steps and decodes RGB, DensiTok completes a four-channel latent in one step for a Gaussian scene.Other 3D-aware approaches use latents decoded into neural fields, pixel-aligned Gaussians, or 3DGS.Source: §2.3 · pp. 3–4

3.1 PROBLEM FORMULATION

• DensiTok generates tokens for unobserved virtual views so frozen reconstruction heads can process a full-view representation from only n observed images.The target is what the geometry encoder would have produced had all N images been available.Source: §3.1 · p. 4

3.2 VAE WITH LEVEL-ADAPTIVE MODULATION

• LAM-VAE fuses each token’s L feature levels adaptively, compresses them into a compact latent, and adds a level-wise residual to the posterior mean.The shared VAE and generator handle all levels, while level contributions can vary across backbones and scenes.Source: §3.2 · pp. 4–5

3.3 DIT FOR GEOMETRY LATENT AUGMENTATION

• A camera-conditioned DiT completes latents for unobserved views from observed latents, returning a full tensor while preserving the observed latents.Training matches generated latents to those encoded from all views; at inference, the unobserved views start from noise.Source: §3.3 · p. 6

3.4 CAMERA CONDITIONING AT TRAINING AND INFERENCE

• At inference, unobserved-view cameras are interpolated along the path through predicted observed cameras, with centers using a natural cubic spline and rotations spherical interpolation.For training, ground-truth cameras are rescaled to match the predicted-camera scale before conditioning the DiT.Source: §3.4 · p. 7

3.5 OVERALL TRAINING STRATEGY

• Training proceeds in three stages: VAE token reconstruction, DiT flow matching on unobserved views, then one-step sampling fine-tuning, with the geometry encoder frozen throughout.The Gaussian head may remain frozen or be minimally finetuned during Stage 2.Source: §3.5 · p. 7

4.1 SETUP

• Experiments train separately on RealEstate10K and DL3DV-10K using ordered N = 16 sequences, and evaluate nested n = 2, 3, 5, and 7 input-view sets.Source: §4.1 · pp. 7–8
• Evaluation measures rendering, camera pose, and depth; depth uses MoGe-2 predictions as pseudo-labels because neither dataset provides official ground-truth depth.Source: §4.1 · p. 8

4.2 RESULTS

• DensiTok improves all three appearance metrics and depth across evaluated sparse-view settings; at n = 3, AnySplat surpasses its N = 16 reference on both datasets.Source: §4.2 · p. 8
• With n = 2 inputs, DensiTok improves alignment, geometry, and RGB quality, filling gaps and occluded regions into more coherent surfaces with clearer object boundaries.Source: §4.2 · pp. 8–9, Figure 3 · p. 8

4.3 ABLATION STUDIES

• With n = 2, level-adaptive fusion and the level-wise residual each improve over uniform fusion, while combining them gives the best results across metrics on both datasets.Source: §4.3 · p. 9, Table 2 · p. 9
• With n = 2, increasing total views N generally improves reconstruction but raises runtime and peak GPU memory; gains diminish beyond N = 12.Source: §4.3 · p. 9, Table 3 · p. 9

• The evaluation does not establish reconstruction quality for extrapolated viewpoints beyond the observed camera interval.Source: §5 · p. 9

• Across three backbones on RealEstate10K and DL3DV, DensiTok improves rendering, camera, and depth accuracy, with sparse inputs surpassing dense-view references in some settings.Source: §6 · p. 10

A.1 FRAME AND GLOBAL DIT BLOCKS

• Each DiT pair applies frame attention within individual views, then global attention across all views, where noisy unobserved views gather evidence from clean observed views.Source: §A.1 · p. 14, Figure 4 · p. 14

A.2 APPEARANCE FEATURE FUSION FOR THE GAUSSIAN HEAD

• For unobserved views, predicted cameras and depths warp observed-image colors into a prior; validity checks reject occluded samples, and no unobserved RGB is needed.Source: §A.2 · pp. 14–15
• Invalid warped pixels use a learned missing-view embedding, and the image embedder and embedding are finetuned under rendering losses because the pretrained head expects complete images.Source: §A.2 · p. 15

A.3 MODEL CONFIGURATIONS

• The shared configurations use latent width dz = 4 by default and dz = 16 for the latent-channel ablation; LAM-VAE selects four backbone-specific feature levels.Source: §A.3 · p. 15

B TRAINING AND EVALUATION DETAILS

• The VAE reconstructs multi-level geometry tokens while the geometry encoder and pretrained reconstruction heads remain frozen.Its objective combines token reconstruction and KL terms with losses that preserve the backbone’s rendered images, cameras, and depths.Source: B TRAINING AND EVALUATION DETAILS · pp. 15–16
• DiT training keeps observed latents clean and interpolates unobserved latents between full-view latents and noise, supervising predicted velocity only on unobserved views.The geometry encoder, VAE, and remaining pretrained head weights stay frozen; the Gaussian head’s image embedding layer and missing-view embedding are finetuned or learned.Source: B TRAINING AND EVALUATION DETAILS · p. 16
• Completed latents are decoded into geometry tokens and passed through the reconstruction heads for image-based supervision at randomly selected views.Camera and depth losses compare completed-token predictions with teacher predictions from the original full-view tokens.Source: B TRAINING AND EVALUATION DETAILS · p. 16
• One-step finetuning fixes unobserved views at t = 1 and uses one explicit Euler step to produce completed latents.The same loss terms and weights are retained, and rendering losses supervise the one-step reconstruction end to end.Source: B TRAINING AND EVALUATION DETAILS · pp. 16–17
• Evaluation samples 80 sequences per dataset, uses four separate novel views per sequence, and tests nested input subsets of n ∈ {2, 3, 5, 7} from N = 16 views.Training configurations are shared across the main AnySplat experiments; the evaluation protocol stratifies intervals by image overlap and samples views separately at each resolution.Source: B TRAINING AND EVALUATION DETAILS · p. 17, Table 5 · p. 17

C.1 RESULTS WITH DA3 BASELINE

• DensiTok improves DA3’s PSNR, SSIM, and LPIPS across both datasets and all evaluated sparse-view counts; it can surpass dense-view appearance results, while depth and camera comparisons have setting-specific qualifications.At larger input counts it also exceeds the dense-view reference in both depth metrics on both datasets, but camera AUC remains lower on DL3DV; runtime stays below the dense-view reference.Source: §C.1 · pp. 17–18, Table 6 · p. 18

C.2 ADDITIONAL QUALITATIVE RESULTS

• In the illustrated examples, DensiTok reduces RGB blur and produces more coherent depth regions at n = 2 and n = 3, with sharper details and less local depth noise at n = 5 and n = 7.The changes include clearer contours around monitors and train seats, and are particularly visible in architectural and furniture examples with DA3.Source: §C.2 · p. 18

C.3 VISUALIZATION OF GAUSSIAN RECONSTRUCTIONS AND CAMERA POSES

• DensiTok’s Gaussian reconstructions show more coherent scene structure and clearer object boundaries than the original AnySplat backbone.Figure 7 pairs each scene with camera visualizations, with ground truth in red and predictions in blue.Source: §C.3 · p. 18
• DensiTok’s predicted camera positions and viewing orientations align more closely with ground truth in the illustrated examples.The reduced positional displacement and angular mismatch are particularly visible in the outdoor example, consistent with camera AUC improvements reported in Table 1.Source: §C.3 · p. 18
• The camera and Gaussian improvements reflect their dependence on shared geometry tokens, from which the frozen camera head decodes final poses.Predicted and interpolated cameras condition latent generation, while final output cameras are predicted again from enriched tokens.Source: §C.3 · pp. 18–19

C.4 COMPARISON WITH SPARSE-VIEW FEED-FORWARD GAUSSIAN SPLATTING MODELS

• The sparse-model comparison evaluates supported input counts at 256 × 256 and distinguishes methods by whether they require externally supplied camera intrinsics or extrinsics.DensiTok, Splatt3R, and FreeSplatter require neither; combinations unsupported by a method’s official implementation are not evaluated.Source: §C.4 · pp. 20–21, Table 7 · p. 22
• All three DensiTok variants outperform FreeSplatter in appearance and depth metrics across nearly all evaluated sparse-view settings, with stated exceptions and metric leaders varying by dataset and input count.At n = 3, all three also outperform NoPoSplat in every reported appearance and depth metric on both datasets; the comparison notes that differing view sampling can cause slight discrepancies from original-paper results.Source: §C.4 · pp. 22–23, Table 7 · p. 22

C.5 COMPARISON WITH DIFFUSION-BASED NOVEL VIEW SYNTHESIS MODELS

• Across matching input counts, DensiTok variants achieve higher PSNR and SSIM than evaluated diffusion baselines on both datasets; LPIPS leadership varies by setting.The comparison uses 512 × 512 evaluation and closely spaced inputs, unlike the widely spaced views often used by diffusion-based methods; this can modestly change results from original papers.Source: Table 8 · p. 23, §C.5 · pp. 24–25
• DensiTok completes inference in under a second, with reported n = 2 runtime of 0.510 to 0.579 seconds versus 76.824 seconds for GLD.The reported speedups over GLD are approximately 133× to 151× across the three backbones at n = 2.Source: Table 8 · p. 23, §C.5 · p. 25

D ADDITIONAL ABLATION STUDIES

• Additional ablations use AnySplat as the common reconstruction backbone; Table 10 specifies a one-step finetuning comparison with dz = 4, n = 2, and N = 16.The table caption identifies flow matching, number of DiT forward evaluations, and runtime as reported quantities.Source: D ADDITIONAL ABLATION STUDIES · p. 26, Table 10 · p. 26

D.1 GENERALIZATION TO UNSEEN DATASETS

• Around one to three decibels of PSNR improvement is typical across ETH3D, ScanNet, and Mip-NeRF 360, with gains also in camera and depth accuracy.These datasets were excluded from DensiTok training; Mip-NeRF 360 depth evaluation uses MoGe-2 pseudo-labels because ground-truth depth is unavailable.Source: §D.1 · p. 26

D.2 EFFECT OF FLOW-MATCHING PRETRAINING AND ONE-STEP FINETUNING.

• Approximately 19× speedup comes from one-step finetuning, which retains most 50-evaluation rendering quality and beats same-cost regression and scratch-training baselines.The comparisons include deterministic regression and one-step training from scratch without flow-matching pretraining; both retain the camera-conditioned DiT, pretrained backbone, and Stage 1 LAM-VAE.Source: §D.2 · pp. 26–27

D.3 EFFECT OF LATENT CHANNEL COUNT

• The 16-channel latent variant improves most appearance, camera, and depth metrics over 4 channels, though not uniformly across metrics.The comparison uses AnySplat with N = 16 and one-step inference on RE10K and DL3DV.Source: §D.3 · p. 27, Table 11 · p. 27

D.4 EFFECT OF GAUSSIAN HEAD FINETUNING

• A fully frozen Gaussian head performs close to image-embedding finetuning, while full-head finetuning yields modest appearance and depth gains with broadly similar camera accuracy.The frozen-versus-image-embedding gap in appearance and depth becomes negligible as observed-view count increases; the authors attribute this to closer view spacing improving warped-color-prior coverage.Source: §D.4 · pp. 27–28, Table 12 · p. 27