Source-linked AI summary
RouteBridge: Reliability-Routed Bidirectional Distillation Between Neural Radiance Fields and 3D Gaussian Splatting
YuanHang Wang, Xin Cao
TL;DR
Existing distillation commonly fixes one representation as a global teacher, despite local disagreements between NeRF and 3DGS. RouteBridge routes supervision per ray using photometric and geometric reliability, and its exports reach 28.56 and 28.77 dB on mip-NeRF 360 while adaptive routing and geometric targets improve results. It retains independent deployable branches, but requires two representations and relies on heuristic reliability scores.
Problem
Existing cross-representation distillation typically fixes one representation as teacher for the entire scene, although local reconstruction reliability can differ between NeRF and 3DGS.
Method
RouteBridge independently warms up NeRF and 3DGS, then routes color, opacity, and normalized-depth supervision per ray from the locally stronger branch or abstains.
Results
RouteBridge reaches 28.56 dB for NeRF and 28.77 dB for 3DGS on mip-NeRF 360, with matched-budget ablations confirming benefits from adaptive routing and geometric ray targets.
Takeaways & Limitations
NeRF and 3DGS can act as local teachers for one another without shared features, an auxiliary conversion network, or additional deployment parameters.
Takeaways & Limitations
Training two representations increases memory, reliability scores are heuristic rather than calibrated uncertainty, and shared errors may evade abstention.
Abstract
from arXiv · showhide
Neural radiance fields (NeRFs) and 3D Gaussian Splatting (3DGS) encode a scene with complementary inductive biases, but existing cross-representation distillation typically fixes one representation as teacher for the entire scene. A globally fixed teacher can propagate local reconstruction errors. We present RouteBridge, a bidirectional framework that selects the teaching direction for each ray. Its reliability estimator combines photometric residuals with representation-specific geometric evidence and routes supervision from NeRF to 3DGS, from 3DGS to NeRF, or abstains. A renderer-independent interface transfers color, opacity, and normalized depth without shared features or point correspondence. On mip-NeRF 360, the NeRF and 3DGS exports reach 28.56 and 28.77 dB, respectively. The 3DGS export improves over 3DGS by 1.56 dB and over NeRF-GS by 0.45 dB while reducing LPIPS to 0.207. On static three-view DTU, RouteBridge obtains 21.12 dB. Ablations show that both adaptive routing and geometric ray targets contribute to the improvement.
1. Introduction
NeRF and 3DGS use different spatial priors and rendering models, motivating RouteBridge’s ray-level choice of teaching direction based on local reliability.
- NeRF uses a continuous density-and-color field with volume rendering, whereas 3DGS uses explicit anisotropic primitives and rasterization.
- Prior work enables conversion or cooperation between NeRF and 3DGS, but does not determine which representation should teach when local predictions disagree.
- RouteBridge warms up independent NeRF and 3DGS branches, then routes color, opacity, and depth supervision from the locally stronger branch or abstains.
- The branches share neither parameters nor feature backbones, require no learned conversion network, and either trained branch can be exported directly.
- 28.77 dB improves the 3DGS export from 27.21 dB on mip-NeRF 360, while adaptive routing exceeds a fixed NeRF teacher by 0.16 dB with lower perceptual error.
2. Related Work
Related work spans representation conversion, joint optimization, mutual teaching, and representation-specific stabilization, while RouteBridge focuses on ray-level selective teaching between heterogeneous renderers.
- Prior methods distill across NeRF architectures, convert between NeRF and 3DGS, or use one representation to initialize or supervise the other.
- NeRF-GS jointly optimizes both representations and shares continuous spatial features, distinguishing it from frozen-teacher approaches.
- Mutual learning permits reciprocal teaching, but RouteBridge selects a direction or abstains per ray because the renderers can fail differently.
- Earlier transport methods use rendered or spatial targets to move information into octrees, feature grids, small MLPs, polygons, points, or hybrid regions.
- Different stabilization methods address sparse-view fields and poorly initialized Gaussians with depth, semantic, entropy, frequency, patch, propagation, or density-control constraints.
- Rendering and representation studies examine finite pixel support, surface alignment, opacity and depth cues, sorting artifacts, and resource constraints as distinct stress tests.
3. Preliminaries
The preliminaries describe NeRF and Gaussian branches with different internal parameters but comparable rendered ray quantities for image-space alignment.
- The NeRF branch maps sampled points and viewing directions to density and color for volume rendering.
- Both branches expose rendered color, opacity, and expected depth through weighted summaries, enabling a common alpha-compositing interface.
- Instant-NGP hash encoding and small decoders are used for the neural-field implementation.
- The Gaussian branch stores means, covariances, opacities, and appearance coefficients, projects them to 2D footprints, and rasterizes sorted contributions.
- The renderer-independent interface compares normalized depth and other rendered quantities on rays without requiring point correspondence or equal model topology.
4. RouteBridge
RouteBridge trains independent NeRF and 3DGS branches with ray-level reliability routing, using photometric and geometric evidence to select supervision or abstain. Distillation transfers renderer-independent color, opacity, and normalized depth targets without shared features or point correspondence.
- Ray reliability and routing: A router combines each branch’s photometric residual with a representation-specific geometric cue to estimate ray reliability.Both cues are clipped using training-ray percentiles, and the reliability score is heuristic rather than calibrated uncertainty.
- Ray reliability and routing: For each ray, detached gates select NeRF-to-3DGS supervision, 3DGS-to-NeRF supervision, or abstention when both branches are weak or similarly reliable.The abstention rule avoids interpreting agreement alone as correctness.
- Ray-space bidirectional distillation: The two renderers exchange color, opacity, and normalized expected depth through a ray-space interface without matching intermediate features or point correspondence.The interface is renderer-independent and operates on comparable rendered quantities despite incompatible parameters.
- Training objective and deployment: Each branch retains its captured-image objective while the routed loss is added after independent warm-up, with the same sampled rays used for observation and transfer losses.Training uses 5k warm-up steps followed by 25k routed joint steps.
- Training objective and deployment: RouteBridge retains ordinary independent NeRF and 3DGS parameterizations, and either original branch can be exported because the router adds no deployment parameters.The branches share no encoding or Gaussian attributes.
5. Experiments
RouteBridge is evaluated on dense-view and sparse-view reconstruction against canonical, stronger, and joint baselines, with matched-budget ablations isolating routing and geometric targets. It improves both exports on mip-NeRF 360, reaches 21.12 dB on three-view DTU, and remains scoped to static RGB novel-view synthesis.
- Protocol: RQ1 evaluates independently deployable NeRF and 3DGS branches against representative baselines on mip-NeRF 360 and static three-view DTU.The evaluation uses all nine mip-NeRF 360 scenes and 15 static three-view DTU scenes, with no test image used for training or routing.
- Main results: 28.77 dB: the Gaussian export exceeds NeRF-GS by 0.45 dB and original 3DGS by 1.56 dB on mip-NeRF 360.It also achieves the best SSIM of 0.827 and lowest LPIPS of 0.207.
- Main results: 28.56 dB: the field export exceeds Instant-NGP (big) by 2.97 dB on mip-NeRF 360.Both independently deployable branches benefit from joint training, while the Gaussian branch retains a small final image-quality advantage.
- Sparse views: 21.12 dB: RouteBridge improves over 3DGS by 1.72 dB, 2DGS by 0.42 dB, and SplatFields by 0.05 dB on static three-view DTU.The smaller margin over SplatFields suggests implicit spatial regularization remains particularly effective with only three input views.
- Ablations: Under identical ray budgets, fixed NeRF teaching reduces PSNR from 28.77 to 28.61 dB and increases LPIPS from 0.207 to 0.213.RGB-only transfer causes a 0.20 dB drop and raises LPIPS to 0.214, supporting the value of adaptive routing and opacity/depth targets.
- Scope: The evaluation targets static RGB novel-view synthesis, excluding temporal, dynamic, semantic-feature, and surface-recovery problems from its experimental claim.The scope passage distinguishes these settings as different problems rather than interchangeable variants.
6. Discussion
The discussion interprets RouteBridge as exploiting complementary local errors rather than assuming equivalent representations, while identifying practical limitations and evaluation cautions.
- Interpretation: Per-ray teaching selection is more effective than imposing one global direction, according to the fixed-NeRF-teacher ablation.The discussion frames routing as complementary to, rather than a replacement for, representation-specific regularization.
- Limitations: RouteBridge requires more memory, uses heuristic rather than calibrated reliability scores, and may miss errors shared by both branches.Image losses also cannot recover structure absent from all observations.
- Evaluation caveats: Very small dense-view margins should not be overinterpreted because many Table 2 baselines come from corresponding papers rather than unified reimplementations.Sorting-aware rasterization and view-dependent Gaussian attributes may also alter reliability cues.
7. Conclusion
RouteBridge connects independently trained neural fields and Gaussian primitives through ray-level reliability routing and renderer-independent distillation. It improves both exports on mip-NeRF 360 and supports gains from adaptive routing and geometric targets under matched budgets.
- Conclusion: RouteBridge keeps NeRF and 3DGS independent while routing color, opacity, and depth distillation at ray level.It requires neither shared features nor an auxiliary conversion network or additional deployment parameters.
- Conclusion: 28.56 dB and 28.77 dB: RouteBridge reaches these values for its NeRF and 3DGS exports on mip-NeRF 360.The static three-view DTU result reaches 21.12 dB.
- Conclusion: Matched-budget ablations confirm that adaptive routing and geometric ray targets both improve the Gaussian branch.The conclusion identifies NeRF and 3DGS as local teachers for one another without shared features or extra deployment components.