Source-linked AI summary
K-Planes: Explicit Radiance Fields in Space, Time, and Appearance
Sara Fridovich-Keil, Giacomo Meanti, Frederik Warburg, Benjamin Recht, Angjoo Kanazawa
TL;DR
Radiance fields need representations that avoid the prohibitive cost of direct 4D storage while extending beyond static 3D factorizations. K-planes uses dimension-pair planes with multiplicative features and a learned linear decoder to represent arbitrary-dimensional scenes. It supports static, dynamic, and varying-appearance reconstruction with compact models, fast optimization, and competitive performance.
Problem
Direct 4D volume storage is prohibitively expensive, and existing static 3D factorizations do not easily extend to higher-dimensional volumes.
Method
K-planes represents a d-dimensional radiance field with planes for every pair of dimensions, combines plane features multiplicatively, and decodes them with a learned-basis linear decoder.
Results
K-planes achieves competitive performance across static, dynamic, and varying-appearance reconstruction while using 200 MB instead of more than 300 GB for a same-resolution 4D volume.
Takeaways & Limitations
The planar factorization provides an interpretable explicit framework that naturally spans static 3D, dynamic 4D, and higher-dimensional radiance fields.
Takeaways & Limitations
Reported SSIM comparisons use different implementations across prior work, with the paper making a best effort to match the relevant implementation for each dataset.
Abstract
from arXiv · showhide
We introduce k-planes, a white-box model for radiance fields in arbitrary dimensions. Our model uses d choose 2 planes to represent a d-dimensional scene, providing a seamless way to go from static (d=3) to dynamic (d=4) scenes. This planar factorization makes adding dimension-specific priors easy, e.g. temporal smoothness and multi-resolution spatial structure, and induces a natural decomposition of static and dynamic components of a scene. We use a linear feature decoder with a learned color basis that yields similar performance as a nonlinear black-box MLP decoder. Across a range of synthetic and real, static and dynamic, fixed and varying appearance scenes, k-planes yields competitive and often state-of-the-art reconstruction fidelity with low memory usage, achieving 1000x compression over a full 4D grid, and fast optimization with a pure PyTorch implementation. For video results and code, please see https://sarafridov.github.io/K-Planes.
1. Introduction
K-planes addresses the difficulty of extending radiance-field factorizations to higher-dimensional scenes with an interpretable, compact, and fast representation. It uses dimension-pair planes, multiplicative features, and a learned linear decoder to support varied scene types while retaining reconstruction quality.
- 4D volumes are prohibitively expensive to store directly, while existing 3D factorizations do not easily extend to higher dimensions.
- The 4D factorization separates spatial structure from space-time changes, making dynamic objects visible in space-time planes and enabling dimension-specific priors.
- K-planes represents a d-dimensional scene with every pair of dimensions, using six planes for 4D and three tri-planes for 3D.The factorization is intended to avoid unnecessary planes while preserving pairwise dimensional interactions.
- Multiplying plane features and using a learned-basis linear decoder makes k-planes white-box while maintaining reconstruction quality competitive with or better than prior black-box models.The linear decoder can replace an MLP decoder when the plane features are multiplied and can model view-dependent color and variable appearance.
- 200 MB represents a 4D volume requiring more than 300 GB at the same resolution, a compression rate of three orders of magnitude.K-planes also trains orders of magnitude faster than prior implicit models and on par with concurrent hybrid models without custom CUDA kernels.
- K-planes is presented as a white-box model for arbitrary-dimensional radiance fields spanning static, dynamic, and variable-appearance scenes.The paper reports competitive performance across reconstruction quality, model size, and optimization time without custom CUDA kernels.
2. Related Work
Related work reduces radiance-field memory or optimization costs through explicit grids, spatial decompositions, and hybrid MLP models, but extending these approaches efficiently and interpretably to dynamic and higher-dimensional scenes remains difficult. K-planes provides a unified planar framework for static, dynamic, and varying-appearance reconstruction.
- Fully implicit NeRF-style models can be slow and black-box, motivating geometric and explicit representations that reduce optimization time.Plenoxels reduced optimization from hours to minutes, but explicit grids grow exponentially with dimension and become intractable for 4D volumes.
- Table 1 characterizes k-planes as explicit, compact, and fast across static, varying-appearance, and dynamic tasks.Here, fast means optimization within fewer than 6 hours on one GPU, compact means roughly fewer than 100 million parameters, and explicit means no MLPs.
- Hybrid methods combine explicit geometric structure with small MLP decoders, while Instant-NGP and TensoRF improve compactness and speed through hashed grids or tensor decompositions.
- K-planes uses an explicit six-plane factorization with multiscale interpolation, feature combination, decoding, volumetric rendering, and spatial-temporal regularization.The representation separates spatial and spatiotemporal variations in 4D.
- Existing spatial decompositions offer memory efficiency and optimization speed for static scenes, but their extension to 4D volumes is not obviously memory-efficient.K-planes defines a unified factorization for 3D and 4D volumes that extends to higher dimensions.
- Dynamic radiance-field methods commonly model deformation over a canonical field or learn a time-conditioned field, with some approaches remaining fully implicit or partially explicit with MLP components.
- K-planes also extends to unbounded environments with varying appearance by adding a global appearance code that disentangles appearance from geometry.
3. K-planes model
K-planes factorizes d-dimensional scenes into pairwise 2D planes, with a six-plane hex-planes instance for 4D scenes. Hadamard feature products, dimension-specific regularization, multiscale planes, and learned linear decoding make the representation expressive, interpretable, and compact.
- K-planes factorization: K-planes represents a d-dimensional scene with planes for every pair of dimensions, yielding tri-planes in 3D, hex-planes in 4D, and deca-planes in 5D.For 4D scenes, the six planes are xy, xz, yz, xt, yt, and zt.
- Hex-planes: The 4D hex-planes representation uses three space-only and three space-time planes, each storing features over spatial and temporal coordinates.The space-only planes are Pxy, Pxz, and Pyz; the space-time planes are Pxt, Pyt, and Pzt.
- Feature combination: Hadamard multiplication combines interpolated plane features into spatially localized signals, unlike addition, which produces less localized intersecting structures.The resulting feature vector is decoded into density and view-dependent color.
- Feature combination: Multiplication substantially improves rendering for linear decoders and modestly improves MLP decoders, allowing the linear decoder to focus on view-dependent color.The ablation reports a large PSNR improvement for the explicit model, while the hybrid model can partially compensate through its MLP.
- Dimension-specific priors: Space-time planes support temporal smoothness and static-transient separation through Laplacian and ℓ1 regularization, while multiscale planes encourage spatial coherence.Space-time features are initialized at the multiplicative identity so unchanged content can remain fixed at 1.
- Feature decoders: A learned color basis replaces fixed spherical harmonics in the linear decoder, retaining feature interpretability while adapting its basis to each scene.A small MLP maps view direction to red, green, and blue basis vectors; density uses a learned view-independent basis.
4. Results
K-planes is evaluated across static, dynamic, and variable-appearance scenes, with competitive reconstruction quality and efficient optimization. The experiments also examine explicit space-time decomposition and appearance control.
- Evaluation scope: K-planes is evaluated on static scenes, dynamic scenes, and Phototourism scenes with variable appearance.The evaluation covers bounded and unbounded static scenes, forward-facing multiview and monocular dynamic scenes, and unconstrained landmark photographs.
- Static scenes: The explicit and hybrid models perform similarly to recent methods on synthetic static scenes, while the hybrid model slightly exceeds the explicit model on LLFF.On LLFF, both versions match or exceed prior state of the art, with the hybrid version achieving slightly higher metrics.
- Dynamic scenes: Both models outperform D-NeRF in quality metrics and training time but do not surpass TiNeuVox and V4D on the D-NeRF benchmark.The benchmark contains eight synthetic videos with one training image per timestep and teleporting camera viewpoints.
- Dynamic scenes: K-planes methods achieve similar quality metrics to prior state-of-the-art on DyNeRF, with the hybrid model achieving higher quality metrics.The comparison concerns real, unbounded, forward-facing multiview videos and includes MixVoxels.
- Space-time decomposition: The planar decomposition exposes static and dynamic scene components by setting time planes to the multiplicative identity and subtracting the static rendering from the full rendering.The decomposition is illustrated for flame salmon and jumping jacks videos.
- Space-time decomposition: The time plane highlights motion regions and learns sparse features corresponding to moving hands and cooking tools.The observed motion differs between flame salmon, where one hand moves, and cut beef, where both hands move.
- Variable appearance: A 32-dimensional appearance code captures global appearance changes while appearance interpolation alters visual appearance without changing geometry.Only the color decoder receives the appearance code, so interpolation affects color rather than density geometry.
5. Conclusions
The paper concludes that k-planes provides a simple, explicit decomposition for radiance fields across dimensions and scene types. It combines compactness, fast optimization, and competitive performance without custom CUDA kernels.
- Conclusions: K-planes decomposes d-dimensional spaces into planes that scale gracefully in model size and optimization time without custom CUDA kernels.The conclusion presents the decomposition as applicable to static 3D, dynamic 4D, and variable-appearance reconstruction.
- Conclusions: The method applies to static 3D scenes, dynamic 4D videos, and unconstrained scenes with a global appearance code.The paper describes this as a unified application across varied reconstruction tasks.
- Conclusions: K-planes achieves competitive performance across varied tasks while maintaining an explicit model, compact size, and fast optimization time.The conclusion characterizes these properties as jointly demonstrated across the studied settings.
6. Appendix
The appendix describes the rendering formulation, evaluation reporting, and ablations used to analyze k-planes design choices. These materials cover metrics, implementation comparisons, multiscale features, feature length, and temporal regularization.
- Volumetric rendering: K-planes uses the standard NeRF volume-rendering formula, summing sample colors weighted by transmission and absorption along each ray.The density, color, and inter-sample distance determine each sample’s contribution.
- Evaluation: The appendix reports PSNR, SSIM, training time, and parameter count across static, dynamic, and variable-appearance datasets.Phototourism uses MS-SSIM instead of SSIM, and timings are based on a single NVIDIA A30 GPU.
- Ablations: Multiscale ablations show that adding lower-resolution planes improves quality, while lower scales alone reduce model size with some quality loss.The experiment is performed on the static Lego scene while holding total feature dimension fixed in one comparison.
- Ablations: Increasing feature length consistently improves quality but linearly increases model size and optimization time.The tradeoff is measured on the static Lego scene using three spatial scales.
7. Model hyperparameters
The appendix documents released configurations and ablations of scale, feature length, and temporal smoothness, alongside full benchmark results and implementation comparisons. These materials also record substantial optimization-time differences against selected baselines.
- Multiscale settings: Including lower-resolution scales improves performance, while using lower scales alone substantially reduces model size with somewhat lower quality than mixed resolutions.This multiscale ablation is conducted on the static Lego scene.
- Feature length: Increasing feature length improves quality with a corresponding linear increase in model size and optimization time.The main experiments use a mixture of M = 16 and M = 32 features across three scales.
- Temporal regularization: A temporal smoothness weight of 0.01 performs best, with PSNR degrading gradually under over- or underregularization.The ablation uses the Jumping Jacks scene with four scales and 32 features per scale.
- Benchmark results: The appendix provides full results for static synthetic, static forward-facing, monocular dynamic, and multiview dynamic scenes.The dynamic monocular benchmark is the synthetic D-NeRF teleporting-camera setting.
- Optimization time: K-planes optimizes dynamic multiview scenes in less than 4 GPU hours, compared with approximately 1344 GPU hours for DyNeRF.The comparison is reported for the full multiview dynamic results.
- Optimization time: Phototourism results require about 35 GPU minutes, whereas NeRF-W trains with 8 GPUs for approximately 384 GPU hours.The comparison concerns the reported full Phototourism results.