ACORN: Adaptive Coordinate Networks for Neural Scene Representation

1 INTRODUCTION

• Existing neural representations often trade computational efficiency for memory efficiency, limiting their ability to scale to large or complex scenes.Explicit representations store many features; coordinate networks require a full network pass for each queried coordinate.Source: §1 · p. 2
• ACORN maps coordinates to local feature grids with a coordinate encoder, then uses a lightweight decoder to evaluate points within each block.The encoder performs the bulk of the computation once per block, reducing the overhead of repeated queries.Source: §1 · p. 2
• An online integer linear program periodically adapts block scales and locations using training loss, allocating network capacity across a multiscale partition.The partition is quadtree- or octree-like, and the strategy requires no preprocessing of the input signal.Source: §1 · p. 2
• ACORN fits a 51,200 × 19,456-pixel image at nearly 40 dB PSNR within a few hours and represents 3D shapes faster and better than existing neural representations.The paper attributes these capabilities to computational and memory efficiency, including pruning empty space and adapting network capacity to the target scene.Source: §1 · p. 2
• The introduction identifies a hybrid implicit–explicit architecture, adaptive resource allocation with pruning, and results on large images and complex 3D scenes as the paper’s contributions.Traditional representations are described as scaling with Nyquist sampling requirements and requiring explicit storage of spatio-temporal samples.Source: §1 · pp. 1–2

2 RELATED WORK

• Explicit scene representations can be evaluated quickly but typically require substantial memory, while coordinate networks require a network pass for each continuous-coordinate query.Coordinate networks represent scenes through network parameters rather than a discrete pixel or voxel grid.Source: §2 · p. 3
• Local implicit methods use a shared decoder conditioned on stored cell-level codes or features, creating additional storage and scalability costs.The cited methods estimate codes through optimization or a convolutional encoder; NSVF also stores a single-scale feature volume.Source: §2 · p. 3
• ACORN differs by using a multiscale representation, block-level coordinate evaluation, and no additional feature inputs, while pruning empty space in an optimized manner.These design differences address the memory and computation tradeoffs described for explicit, implicit, and local implicit approaches.Source: §2 · p. 3
• The representation is presented as applicable to 2D images and 3D shapes and as complementary to most neural rendering and graphics approaches, though a detailed application study is out of scope.The 3D shapes are represented as occupancy networks.Source: §2 · p. 4
• The paper situates its multiscale architecture alongside adaptive mesh refinement, bounding-volume hierarchies, and wavelet decompositions.These techniques are cited as related multiscale approaches from graphics and interactive techniques.Source: §2 · p. 4

3.1 Multiscale Block Parameterization

• The input domain is partitioned into active quadtree or octree blocks, with each input value represented by one block at one scale.A block index supplies global position and normalized scale, while a continuous local coordinate describes position within the block.Source: §3.1 · p. 4

3.2 Neural Network Architecture

• A coordinate encoder maps each block index to a feature grid; local coordinates interpolate features that a lightweight decoder maps to signal values, avoiding repeated expensive encoder passes.Feature grids may also be reused for repetitive structures across spatial locations and scales.Source: §3.2 · pp. 4–5

3.3 Online Multiscale Decomposition

• At a fixed block budget, an error-driven ILP periodically merges, keeps, or splits blocks, placing finer blocks in harder-to-fit regions and using coarser blocks for simpler regions at inference.Each block receives a fixed number of training samples, so finer subdivision allocates more samples to the regions that are harder to fit.Source: §3.3 · p. 5

3.4 Pruning

• Blocks with low fitting error and low within-block variance can be pruned and represented by a lookup-table value, freeing partition capacity for finer subdivision elsewhere.The pruning criteria include an application-specific error threshold and low variance among block values.Source: §3.4 · p. 6

4.1 Image Fitting Task

• Image fitting samples random stratified coordinates within active blocks, uses bilinear interpolation for target values, and applies per-block mean-squared error.The block partition is optimized every 500 iterations, with a maximum of 1,024 blocks; pruning is disabled under the assumption that all blocks contain non-zero texture.Source: §4.1 · p. 7
• The image comparisons use approximately parameter-matched SIREN, ReLU with positional encoding, and a fixed-decomposition ACORN ablation.Source: §4.1 · p. 7

4.2 Performance

• ACORN converges about two orders of magnitude faster than state-of-the-art coordinate networks and fits the Pluto image well above 40 dB PSNR.Baselines were evaluated on the same NVIDIA RTX 6000 GPU after 100,000 training iterations.Source: §4.2 · p. 7
• Adaptive block decomposition improves image quality over a fixed decomposition and slightly accelerates training by starting with 64 coarse-scale blocks.The fixed-decomposition model still converges rapidly and outperforms other methods.Source: §4.2 · p. 7
• 19,456 × 51,200 pixels—about three orders of magnitude beyond prior neural image representations—is the resolution of the Tokyo panorama fit.The multiscale result achieves 38.59 PSNR.Source: §4.2 · p. 7, Figure 4 · p. 8

4.3 Interplay between the quadtree decomposition, time and fitting accuracy

• On Pluto, fewer blocks make fitting faster but less accurate; adding blocks improves accuracy until the coordinate encoder cannot generate accurate finest-scale features across the image.Source: §4.3 · pp. 7, 9

5 REPRESENTING COMPLEX 3D SCENES

• The tests fit occupancy fields for a 10-million-polygon Thai statue with intricate reliefs and a 512-thousand-polygon engine with thin features and hard edges.Source: §5 · p. 9

5.1 Occupancy Prediction Task

• Training samples half of its 3D points near the mesh surface and half uniformly within the volume, predicts occupancy, and uses binary cross-entropy loss.SPSR instead takes oriented point clouds as input.Source: §5.1 · p. 9

5.2 Applying Multiscale Model to Occupancy Prediction

• For 3D, ACORN replaces the quadtree with an octree and predicts a dense feature cube; online decomposition allocates samples toward hard-to-fit regions.The method periodically prunes likely empty octants, eliminating sampling there and freeing capacity for more complex regions.Source: §5.2 · p. 9

5.3 Metrics

• Evaluation compares occupancy predictions with ground truth using IoU, precision, recall, and F-score, and compares extracted meshes using Chamfer distance.The reported tables also include model size, training GPU memory, and time to reach the reported results.Source: §5.3 · p. 9

5.4 Evaluating Performance

• ACORN outperforms all baselines on volume and mesh accuracy metrics and requires nearly an order of magnitude less memory than a comparable MLP-based implicit representation at the same parameter count and batch size.The comparison also reports higher representation accuracy for ACORN.Source: §5.4 · p. 9

• The representation itself is smaller than training GPU memory: its size is the parameter count multiplied by encoding size, using 32-bit floats in the experiments.Source: §6 · pp. 9, 11
• Uncompressed ACORN models are smaller than the uncompressed image and 3D data in most cases.The comparison is reported in Table 6.Source: §6 · p. 11, Table 6 · p. 11
• Pruning and quantization are identified as possible ways to further reduce ACORN model size.Source: §6 · p. 11

7 DISCUSSION

• ACORN combines a hybrid implicit–explicit coordinate network with training that adapts capacity to the target scene in an optimized multiscale fashion.Source: §7 · p. 11
• Each block represents one scale, the maximum block count is fixed in advance, and updating the partition requires retraining on new blocks, temporarily increasing fitting loss.Source: §7 · p. 11
• The method does not enforce continuity across blocks, and minor boundary artifacts can occasionally occur despite high-quality fitting near boundaries.Source: §7 · p. 11, Figure 6 · p. 11
• Future work includes neural rendering, fluid simulation, level-of-detail queries, higher-dimensional signals, and network compression.Source: §7 · p. 11

8 CONCLUSION

• Neural signal representations are crucial to emerging techniques in rendering, simulation, imaging, and geometric modeling, yet remain limited compared with traditional graphics representations.Source: §8 · p. 11
• ACORN takes steps toward closing the representation gap between neural networks and traditional graphics representations.Source: §8 · p. 11
• The approach enables neural networks to represent large-scale images and complex shapes efficiently.Source: §8 · p. 11

A.1 Image Fitting

• For Pluto, ACORN uses a 4-layer, 512-unit coordinate encoder producing a 16-channel 32×32 feature grid and a one-layer, 64-unit feature decoder.Training uses 100,000 iterations, a maximum of 1,024 blocks, and a 1×10^-3 learning rate; the block partition is optimized every 500 iterations.Source: §A.1 · pp. 12–13
• Tokyo uses a 64-channel feature grid with dimensions 19×50 and an 8-layer, 2,048-unit coordinate encoder with 10 positional-encoding frequencies.Blocks preserve the 19,456×51,200 image aspect ratio; the maximum is 16,384 blocks, and optimization runs for 57,000 iterations.Source: §A.1 · p. 13

A.2 3D Occupancy Prediction

• For 3D representations, ACORN uses a 4-layer, 512-unit encoder producing an 18-channel 12×12×12 feature cube and a one-layer, 64-unit decoder.The maximum is 1,024 octants; Thai Statue and Engine are optimized for 48,000 and 84,000 iterations, respectively, with 64 points sampled per octant per iteration.Source: §A.1 · p. 13

B.1 Teaser figure: “Lucy” and “Dragon”

• The evaluation covers two teaser shapes: Lucy and Dragon.Source: §B.1 · p. 13, Table 7 · p. 13
• Both ACORN and SIREN are included in the quantitative comparison for the teaser shapes.Source: §B.1 · p. 13, Table 7 · p. 13
• Table 7 presents a quantitative evaluation of ACORN against the SIREN baseline on Lucy and Dragon.Source: Table 7 · p. 13, §B.1 · p. 13

Pluto Image

• The Pluto results include additional insets and error maps comparing ACORN with SIREN and ReLU positional encoding baselines.Source: §1 · p. 14

Gigapixel Image

• Tokyo’s block decomposition assigns larger blocks to flat-texture regions and the finest-scale blocks to detailed areas such as signage text.Source: §1 · p. 14
• After roughly 30 hours of training, the Tokyo image reaches roughly 39 dB PSNR; it reaches 30 dB in less than an hour.Training uses an NVIDIA RTX 8000 GPU.Source: §1 · p. 14

3D scene results

• ACORN shows an advantage in Intersection over Union and convergence speed, although block reshuffling causes short-term IoU drops before improving the representation long term.The reported drops occur during ACORN’s optimization procedure.Source: §2 · p. 14, Figure 4 · p. 16

3D scene representation baseline details

• The reconstruction baselines use different inputs: Convolutional Occupancy Networks use a point cloud and occupancy samples, while SIREN is adapted from SDF prediction to occupancy prediction.SIREN uses binary crossentropy loss and ground-truth occupancy values; its described model has four hidden layers with 2,900 units each.Source: §3 · p. 14
• For Convolutional Occupancy Networks, training uses 2,048 query points per voxel crop, 25 x 25 x 25 voxels per crop, and 50,000 iterations.Each voxel has size 0.01.Source: §3 · p. 14
• SPSR reconstructs a mesh from an oriented point cloud and, unlike ACORN and most other baselines, requires associated normal vectors.The described setup samples 5 million surface points and uses trimming parameter –trim 6.Source: §3 · p. 14
• Mesh extraction samples 20 million surface points, predicts occupancy at a combined resolution of 1024 x 1024 x 1024, and applies Marching Cubes [Lorensen and Cline 1987].The voxel size and crop resolution match training.Source: §3 · p. 14

3D scene results

• ACORN captures large-scene geometric details better than other neural methods and matches SPSR, which requires normal vectors as input.The comparison is reported for the engine model.Source: Figure 5 · p. 17