Source-linked AI summary
SPLATIFY: Reproduce, Discover, Innovate! From Papers and Ideas to Trainable 3DGS Code
Seemandhar Jain, Keshav Gupta, Manmohan Chandraker
TL;DR
Rapid growth in 3DGS research makes it difficult to absorb new methods, and converting papers into trainable implementations remains a substantial effort. SPLATIFY combines grammar-constrained synthesis, dependency recovery, code refinement, and knowledge-driven discovery. It reports expert-level reproduction results on papers without public code and gains from composing techniques.
Problem
3DGS research has grown rapidly, while many papers lack code and reimplementing recent methods remains slow and difficult.
Source: §1 · pp. 1–2Method
SPLATIFY uses a CFG-constrained gsplat template, fork-aware recovery, structured code generation and visual feedback, then composes techniques or retrieves external physical priors.
Source: §1 · p. 2, §3.2 · p. 3Results
On papers without public code, SPLATIFY matches expert implementations within ±0.5 dB PSNR and ±0.02 SSIM, while existing baselines fail to produce trainable code.
Source: §1 · p. 2Takeaways & Limitations
SPLATIFY supports reproduction, compositional improvement, and methods for previously unaddressed scene types without human guidance.
Source: §1 · p. 2
Abstract
from arXiv · showhide
The rapid growth of 3D Gaussian Splatting (3DGS) research demands significant effort to reimplement papers before building on them. We introduce SPLATIFY, a multi-agent framework that converts 3DGS papers into trainable gsplat-based implementations, where generic paper-to-code methods and frontier models fail. SPLATIFY achieves this through five innovations: (1) A context-free grammar for gsplat over a modular method template with extension points for losses, densification, rendering, and optimization, constraining synthesis so generated code satisfies gsplat's architectural invariants by construction. (2) Architectural elements for faithful reproduction: fork-aware citation recovery retrieving component-level code at function-level granularity, Graph-of-Thought synthesis in topological dependency order, RAG-guided in-context example selection from over 20 verified implementations, and visual feedback combining PSNR-guided regeneration, Gaussian-level structural checks, and VLM-driven patching. (3) Knowledge-driven compositional improvement that autonomously finds weaknesses and composes complementary regularizers, losses, and densification strategies to improve upon original results. (4) Interdisciplinary method discovery where agents retrieve physical priors from outside the 3DGS literature and compose them with rendering knowledge to produce methods for previously unaddressed scene types. (5) SPLATIFY-Bench, an evaluation framework across 30 diverse 3DGS papers. On papers without public code, SPLATIFY matches expert implementations while reducing development time from weeks to minutes, and through compositional discovery further improves PSNR by up to 2.4 dB. We additionally demonstrate novel methods for volumetric nebula rendering and other scientific domains, synthesized entirely by SPLATIFY.
1. Introduction
3DGS research has expanded faster than researchers can absorb it, while translating recent methods into working code remains slow. SPLATIFY targets this gap with a framework for reproduction, compositional improvement, and cross-domain method invention.
1. Introduction
- Over 40% of tracked 3DGS papers release no code, and reimplementing recent methods typically takes weeks of manual work and debugging.Generic paper-to-code systems produce outputs that do not compile against rasterization backends, while frontier models lack domain knowledge needed for training.Source: §1 · p. 2
- The field’s knowledge is difficult to discover and compose, leaving manual exploration of the many possible technique combinations negligible.The authors also identify unaddressed applications such as optically thin nebulae and fluorescence microscopy.Source: §1 · p. 2
- SPLATIFY converts 3DGS papers into trainable gsplat plugins and supports compositional improvement and cross-domain method invention.Its five contributions include a CFG-based template, faithful reproduction tools, autonomous composition, physical-prior retrieval, and a 30-paper benchmark.Source: §1 · p. 2
- SPLATIFY matches expert implementations on papers without public code within ±0.5 dB PSNR and ±0.02 SSIM, while existing baselines fail to produce trainable code.The authors report that development time falls from weeks to minutes.Source: §1 · p. 2
2. Related Work
Prior paper-to-code and multi-agent systems do not address 3DGS’s fragmented, tightly coupled implementations. SPLATIFY combines structured generation and iterative feedback with fork-aware recovery and Gaussian-specific checks.
2. Related Work
- 3DGS methods often modify tightly coupled components across repository forks, a setting that domain-agnostic reproduction systems and NERFIFY do not handle.SPLATIFY adds fork-aware recovery from parent-repository diffs, Gaussian structural checks, and PSNR-curve monitoring, then extends beyond reproduction.Source: §2 · p. 2
- Figure 2 gives a simplified overview of reproduction, refinement, and interdisciplinary method discovery; Figure 3 provides the detailed architecture.The figure caption identifies the high-level stages but not their full system architecture.Source: Figure 2 · p. 3
- SPLATIFY combines a gsplat CFG for architectural validity with PSNR training-curve analysis and VLM-guided patching for iterative repair.The paper situates these choices alongside prior structured generation, Graph of Thoughts, and self-refinement approaches.Source: §2 · p. 3
3. Proposed Approach
SPLATIFY formalizes faithful reproduction, compositional improvement, and interdisciplinary discovery as increasingly broad objectives. Its pipeline combines a constrained plugin template, retrieval and dependency recovery, structured code generation, and visual feedback, with a search stage for composing techniques or retrieving external physical priors.
3.1. Problem Statement
- Faithful reproduction requires an executable repository that compiles against gsplat, trains to convergence, implements each novel component, and comes within 0.5 dB PSNR of reported metrics.Compositional improvement validates technique combinations on held-out views; interdisciplinary discovery retrieves physical priors for scene types not previously addressed by 3DGS papers.Source: §3.1 · p. 3
3.2. CFG-Constrained gsplat Method Template
- A CFG and modular gsplat template constrain agents to valid extension points while keeping the core training, rasterization, density-control, and optimizer structure fixed.The template defines nine extension points and supports custom losses or per-Gaussian attributes through selected hooks.Source: §3.2 · p. 3
3.3. Proposed Multi-Stage Framework
- SPLATIFY’s multi-stage framework centers on a knowledge base and verified implementations, then adds fork-aware recovery, code generation, and visual feedback.Figure 3 separates reproduction from knowledge discovery and invention.Source: §3.3 · p. 3, Figure 3 · p. 4, §3.3.1 · p. 4
3.3.1. Stage 1: RAG-Guided In-Context Learning and CFG
- The RAG stage selects verified implementations by extension-point overlap and technique type, and adds each successfully implemented paper to the example library.The library starts with 20 manually validated papers.Source: §3.3.1 · p. 4
3.3.2. Stage 2: Fork-Aware Compositional Dependency Resolution
- Fork-aware recovery follows multi-hop dependencies and diffs known repository forks against their parents to recover changed functions and transitive modifications.When no repository exists, the agent falls back to PDF extraction.Source: §3.3.2 · p. 4
3.3.3. Stage 3: Grammar-Guided Repository Generation
- GoT generation maps contributions to template extension points, verifies the combination, and produces files in topological order with smoke testing.Most repositories pass within two iterations.Source: §3.3.3 · pp. 4–5
3.3.4. Stage 4: Visual-Driven Feedback
- Visual feedback combines local-window PSNR and SSIM maps, cross-view geometry checks, VLM diagnoses, and full training-curve monitoring to target patches.The agent flags late-stage PSNR collapse after roughly 15k iterations and plateaus below 0.1 dB improvement over a 5k-iteration window after warm-up.Source: §3.3.4 · p. 5
3.4. Knowledge Discovery via Evolutionary Technique Composition
- Evolutionary search mutates technique configurations, evaluates candidates through a multi-fidelity cascade, and uses Pareto ranking to guide composition or invention.Discovery searches techniques in the knowledge base; invention also retrieves physical priors, and the reported demonstrations include up to +2.4 dB PSNR and methods for nebulae and thin-film iridescence.Source: §3.4 · p. 5
4. Experiments
The experiments evaluate SPLATIFY on SPLATIFY-BENCH across reproduction quality, novelty preservation, knowledge discovery, and ablations.
4. Experiments
- The evaluation uses SPLATIFY-BENCH, a new benchmark for 3DGS paper-to-code synthesis.The benchmark results are reported for reproduction quality, novelty preservation, knowledge discovery, and ablations.Source: §4 · p. 5
- The experiments include an assessment of reproduction quality.The supplied overview identifies this as one of the result categories.Source: §4 · p. 5
- The evaluation also covers novelty preservation, knowledge discovery, and ablations.The overview does not provide detailed outcomes for these categories.Source: §4 · p. 5
4.1. SPLATIFY-BENCH
SPLATIFY-BENCH evaluates 30 papers in three sets designed to assess reproduction across code-availability conditions and technical novelty. Set 1 pairs papers without public code with commissioned expert references, reducing the risk of code-pretraining contamination.
Benchmark composition
- SPLATIFY-BENCH covers 30 papers across never-implemented, public-code, and novelty-coverage sets.The sets contain 10 papers each; novelty-coverage papers are used exclusively for novelty analysis.Source: §4.1 · p. 5
Set 1: Never-Implemented Papers
- Set 1 pairs 10 papers without public code with commissioned expert references, so corresponding implementations were unavailable for pretraining.The benchmark describes this as ensuring uncontaminated evaluation.Source: §4.1 · p. 5
Set 2: Papers with Public Code
- Set 2 includes 10 papers with released repositories portable to gsplat, enabling direct comparison against original author code.The selected papers span densification strategies, regularization losses, and appearance modeling.Source: §4.1 · p. 5
Set 1: Never-Implemented Papers
- For Set 1, all other baselines failed to produce trainable code for any paper.Table 1 compares SPLATIFY with reported results and human implementations.Source: Table 1 · p. 6
4.2. Baselines, Metrics, and Experimental Setup
The evaluation compares SPLATIFY with generic paper-to-code systems, frontier LLMs, and expert implementations. Methods are assessed on shared training conditions using rendering-quality metrics and Gaussian count.
Baselines
- The comparison includes Paper2Code, AutoP2C, GPT-5.2 ET, Gemini 3.1 Pro, DeepSeek 3.2, and expert implementations.Expert implementations typically required 1–2 weeks per paper.Source: §4.2 · p. 6
Experimental setup
- SPLATIFY uses GPT-5.2 as its base LLM and completes a paper in 10–30 minutes.Baselines receive the same paper PDF and supplementary materials, prompts, and compute budget.Source: §4.2 · p. 6
Metrics and protocol
- Evaluation reports PSNR, SSIM, LPIPS, and Gaussian count across nine Mip-NeRF 360 scenes after 30k training iterations.Every eighth image is held out for testing; training and test viewpoints are identical across methods.Source: §4.2 · p. 6
4.3. Comparison to Expert Implementations
On papers without public code, SPLATIFY approaches expert results while other baselines fail to produce trainable code. On papers with public code, it achieves comparable performance to original repositories under identical training conditions.
4.3.1. Set 1: Never-Implemented Papers
- Expert developers needed one to two weeks per Set 1 paper, while SPLATIFY generated comparable code in minutes.The lack of released implementations also rules out corresponding code exposure during pretraining.Source: §4.3.1 · p. 6
- Within ±0.5 dB PSNR and ±0.02 SSIM of expert implementations on average, SPLATIFY closely matches expert quality in Set 1.The qualitative comparison shows rendered novel views.Source: §4.3.1 · p. 6, Figure 4 · p. 6
- All baselines fail to produce trainable code on Set 1, with generic systems unable to recover implicit citation dependencies or structure multi-file gsplat plugins correctly.Table 2 compares executability.Source: §4.3.1 · p. 6, Table 2 · p. 6
4.3.2. Set 2: Papers with Public Code
- Under identical training conditions, SPLATIFY's Set 2 implementations have average deviations under 0.3 dB PSNR from original author repositories.The methods are translated into standardized gsplat plugins within Nerfstudio.Source: §4.3.2 · p. 6, Table 3 · p. 6
4.4. Novelty Coverage Analysis
The novelty analysis scores whether generated implementations preserve paper components and hyperparameters, with an additional weighted semantic score. SPLATIFY achieves semantic scores above 0.94 across papers, and human expert scoring agrees strongly with the LLM judge.
Scoring metrics
- Novelty coverage is assessed with correct, partial or incorrect, and missing-component labels, plus hyperparameter fidelity W.A weighted semantic score also rates component completeness while recognizing functionally equivalent implementations and penalizing semantic errors.Source: §4.4 · pp. 6–7
Results
- Above 0.94 semantic scores across all papers show SPLATIFY's near-perfect novelty-preservation scores.The passage reports this result in Table 4.Source: §4.4 · p. 7
Scoring validation
- Human experts agreed with the LLM judge at κ=0.87 on component labels and Spearman ρ=0.92 on semantic scores.Experts independently scored implementations blind to system identity.Source: §4.4 · p. 7
4.5. Knowledge Discovery: Beyond Reproduction
SPLATIFY evaluates knowledge discovery by composing techniques for reproduced methods and creating methods for scene types no published 3DGS paper has addressed. The reported results include gains over reproduction-only baselines and physics-informed methods for nebula and thin-film rendering.
4.5.1. Improving Existing Methods
- Random sampling used 8 times as many tokens and reached only a quarter of the PSNR gain achieved by the discovery loop.Agents independently diagnosed weaknesses, retrieved techniques, and validated the compositions.Source: §4.5.1 · p. 7
4.5.2. Cross-Domain Method Creation
- The cross-domain methods use physical priors from outside 3DGS to address scene types where standard 3DGS fails; both scenarios were human-verified at the code level.Figure 5 compares ground truth, 3DGS, and agent-generated results for Nebula and Bubble; it reports resolved opaque-shell artifacts for NebulaGS and captured iridescent variation for BubbleGS.Source: §4.5.2 · p. 7, Figure 5 · p. 7
- 32.52 dB PSNR and 0.914 SSIM versus 31.19 dB and 0.862 for vanilla 3DGS; NebulaGS also reduces per-view variation without affecting rendering speed.Its six training-time modifications include an opacity ceiling, a per-Gaussian emission parameter, and spectral-frequency supervision.Source: §4.5.2 · p. 7
4.5.1. Improving Existing Methods
- +2.4 dB PSNR is the maximum improvement compositional discovery reports over reproduction-only baselines.The search combines existing techniques rather than inventing new primitives.Source: §4.5.1 · p. 7, Table 5 · p. 8
4.5.2. Cross-Domain Method Creation
- 27.81 dB PSNR and 0.869 SSIM versus 26.97 dB and 0.858 for vanilla 3DGS; BubbleGS improves LPIPS from 0.264 to 0.244, while speed falls from 99.1 to 69.1 FPS.The agents combine six thickness-aware modifications, including a thin-film chrominance loss and a shell-flatness loss.Source: §4.5.2 · p. 8
4.6. Ablation Study
The ablation study identifies components that affect trainability and synthesis quality, including grammar constraints, recovery of borrowed code, smoke tests, and dependency ordering.
4.6. Ablation Study
- CFG constraints raise trainability from 70% without them to 100% with them, while fork-aware recovery is equally critical: without it, 30% of novelties go unimplemented.The ablation averages results over 10 Set 1 papers; Table 6 describes component impacts averaged over 10 SPLATIFY-BENCH papers.Source: §4.6 · p. 8, Table 6 · p. 8
- Smoke tests account for the largest single trainability drop at 50%, with Gaussian structural checks responsible for roughly half.PSNR curve monitoring catches late-stage failures that short runs miss.Source: §4.6 · p. 8
- One-shot generation without Graph-of-Thought scores 0.49 despite high novelty fidelity, as monolithic files break dependency ordering.The supplied results also identify PSNR curve monitoring as a way to catch late-stage failures invisible during short runs.Source: §4.6 · p. 8
5. Conclusion
SPLATIFY converts 3DGS papers into trainable gsplat implementations and reports matching expert implementations while reducing development time. It also reports gains from compositional discovery and working methods for previously unaddressed scene types.
5. Conclusion
- SPLATIFY matches expert implementations within ±0.5 dB PSNR while all baselines fail to produce trainable code, reducing development time from weeks to minutes.The comparison is reported on SPLATIFY-BENCH.Source: §5 · p. 8
- Compositional discovery yields up to +2.4 dB PSNR gain, while cross-domain invention produces working methods for scene types no prior paper has addressed without human guidance.The framework uses grammar-constrained synthesis, fork-aware citation recovery, and closed-loop visual feedback.Source: §5 · p. 8
- Adapting SPLATIFY to another domain requires defining a CFG over that framework’s extension points and curating verified paper-code pairs.The authors state that its architectural elements are not specific to Gaussian Splatting.Source: §5 · p. 8