vMAP: Vectorised Object Mapping for Neural Field SLAM

• Without 3D priors, usually only directly observed object parts can be reconstructed, causing holes and missing parts; CAD or category-level models enable full shape estimation only for covered objects.Source: §1 · p. 1
• vMAP represents each object with a separate tiny MLP and simultaneously optimises many object models on one GPU through vectorised training.The approach is designed for realistic, real-time scanning and often enables watertight reconstruction without 3D priors.Source: §1 · p. 1
• Compared with a single neural field using a similar number of weights, separate object models produce more accurate and complete scene reconstructions.The system maps scenes with up to 50 objects using 40KB of learned parameters per object.Source: §1 · p. 1
• Experiments on simulated and real-world datasets show state-of-the-art scene-level and object-level reconstruction performance, and the representation supports recomposing scenes with new object configurations.Source: §1 · p. 2

• Earlier object-based scene representations ranged from CAD and category-level shape priors to methods without shape priors, which could reconstruct only visible object parts [11] [12] [33] [16] [39].Source: §2 · p. 2
• Neural-field SLAM systems represented whole scenes, while DeRF and KiloNeRF divided scenes into spatial parts; vMAP instead assigns tiny MLPs to semantically meaningful objects [24] [25].Source: §2 · p. 2
• ObjectNeRF and ObjSDF conditioned object representations on masks and learnable codes, but trained offline with object codes entangled in a scene network [40] [38].Source: §2 · p. 2
• Earlier neural-field work studied single-object representations [6] [1] [10], whereas vMAP reports simultaneous real-time training of many individual models across a whole scene.Source: §2 · p. 2

3.1. System Overview

• The system overview presents object-level mapping, efficient vectorised training, improved pixel sampling and surface rendering, and scene recomposition as its pipeline stages.Figure 2 is identified as an overview of the training and rendering pipeline.Source: §3.1 · p. 2, Figure 2 · p. 3

3.2. Vectorised Object Level Mapping

• Object association matches instances across frames using semantic consistency and 3D-bound spatial consistency, while unmatched instances receive new models.Object bounds are estimated from depth and camera pose, then refined as observations accumulate; camera tracking is provided by an external system.Source: §3.2 · pp. 2–3
• Each object uses its own keyframe buffer, allowing its training to stop or resume without interference from other objects.Models of the same design are stacked and trained simultaneously with vectorised operations, improving training speed through more efficient use of GPU resources.Source: §3.2 · p. 3

3.3. Neural Implicit Mapping

• Depth-guided sampling places samples near the observed surface and uniformly between the camera's near bound and that surface to guide geometric learning.When depth is invalid, the surface location is replaced by the far bound; the implementation uses a 3 cm sampling variance.Source: §3.3 · pp. 3–4
• The rendering model omits viewing direction and uses a continuous occupancy field with a binary visibility indicator, excluding transparent objects.Termination probabilities along rays define rendered occupancy, depth, and colour.Source: §3.3 · p. 4
• For each object, training accumulates depth, colour, and occupancy losses across all K objects.Training pixels are sampled inside each object's 2D bounding box, with depth and colour supervision applied to pixels inside its mask.Source: §3.3 · p. 4

3.4. Compositional Scene Rendering

• Objects can be queried within their estimated 3D bounds and manipulated independently; scene rendering ranks object depths along rays for occlusion-aware composition.Ray-box intersections provide near and far bounds for novel-view rendering.Source: §3.4 · p. 4

4.1. Experimental Setup

• The evaluation spans Replica, ScanNet, TUM RGB-D, and self-captured Azure Kinect sequences, using qualitative comparisons and quantitative mesh metrics where ground-truth meshes are available.Object-level metrics average results across objects to address scene-level metrics’ bias toward large objects such as walls and floors.Source: §4 · p. 4, §4.1 · pp. 4, 6

4.2. Evaluation onScene andObjectReconstruction

• More than 50–70% improvement over iMAP and NICE-SLAM for object-level completion is reported on Replica scenes.The evaluation covers eight Replica scenes, with some baselines retrained using ground-truth poses.Source: Table 2 · p. 5, §4.2 · p. 6
• On ScanNet, vMAP produces more coherent objects than TSDF-Fusion, finer details than ObjSDF with much shorter training, and sharper boundaries and textures than NICE-SLAM.The sequence uses noisier ground-truth depth maps and object masks; ObjSDF uses pre-selected posed images and longer offline training.Source: §4.2 · pp. 6–7, Figure 4 · p. 6
• On TUM RGB-D, vMAP generally yields more coherent, hole-filled objects than TSDF-Fusion, but inconsistent detections can split instances and unseen-back completion remains beyond its reach.On a live Azure Kinect tabletop scene, it produces watertight meshes from multiple object categories.Source: §4.2 · p. 7, Figure 7 · p. 7

4.3. Performance Analysis

• With fewer than 1M parameters, vMAP updates maps at about 5 Hz and trains 1.5× faster than iMAP and 4× faster than NICE-SLAM.The comparison uses ground-truth poses, each method’s default hyperparameters, and the same hardware.Source: Table 4 · p. 7, §4.3 · p. 7
• Vectorised training keeps each optimisation step at 15 ms or less for as many as 200 objects, with speed gains narrowing at the hardware memory limit.The speed advantage is especially pronounced with many objects and remains stable across model sizes.Source: §4.3 · p. 8, Figure 8 · p. 8
• Object-level performance begins to saturate at hidden size 16, suggesting the representation can be parameterised accurately with few parameters.Further increases in model size yield minimal or no improvement.Source: §4.3 · p. 8, Figure 9 · p. 8
• The shared-MLP variant slightly degrades reconstruction without distinct speed gains, while entangled object codes and fixed capacity constrain its design.vMAP’s separate object models can be stopped or resumed without inter-object interference.Source: §4.3 · p. 8

5. Conclusion

• vMAP decomposes scenes into meaningful instances represented by separate tiny MLPs, providing an efficient, flexible object-centric representation.The system supports scene recomposition, independent tracking, and continual updating of objects of interest.Source: §5 · p. 8
• The system predicts plausible watertight surfaces for objects even under partial occlusion.Its object-centric reconstructions are described as more accurate and compact.Source: §5 · p. 8
• Completion of fully out-of-view regions remains beyond the system's reach because it lacks general 3D priors.The conclusion also identifies inconsistent off-the-shelf instance masks as a limitation; global constraints are suggested as a better response.Source: §5 · p. 8

A. Interactive Visualisation

• The authors recommend the project website as a place to view selected reconstructions.The cited website is https://kxhit.github.io/vMAP.Source: A. Interactive Visualisation · p. 11
• The visualizations show real-time scene-level reconstructions from selected sequences.The passage also identifies object-level reconstructions on the site.Source: A. Interactive Visualisation · p. 11
• The selected sequences include object-level reconstructions alongside scene-level views.The passage does not specify which sequences are shown.Source: A. Interactive Visualisation · p. 11

Depth-Guided Sampling

• Rays through an object's bounding box that do not belong to its instance are terminated at the object surface to limit impact on occluded objects.Sampling is concentrated near the surface using depth measurements, following an approach similar to ObjectNeRF [40].Source: B. Implement Details and Discussions · p. 11
• Figure A visualizes depth-guided sampling, with sampled points colored by measured depth.This is the visual guide to the sampling procedure described in the text.Source: B. Implement Details and Discussions · p. 11

Object-Level Positional Encoding

• Integrated positional encoding [2] and a scaling factor s bound objects to a unit box in [−1, 1], addressing size-related training bias.The background model uses a slightly larger scaling factor.Source: B. Implement Details and Discussions · p. 11
• A larger scale s yields smoother geometry suited to large objects, while a smaller s suits objects with complex geometry such as chairs.The scale can be object-specific when an object-size prior is known; sofas and cups illustrate larger and smaller choices, respectively.Source: B. Implement Details and Discussions · p. 11

C. Additional Experimental Results on Replica Scenes

• Detailed scene-level and object-level 3D reconstruction results are listed for each Replica sequence in Tables A and B.The passage identifies the tables' scope but provides no per-sequence values.Source: C. Additional Experimental Results on… · p. 11
• Figure B visualizes 3D object reconstructions trained with different scales.The caption does not report which scale performs best.Source: C. Additional Experimental Results on… · p. 12
• The listed Replica sequence names are room-0, room-1, room-2, office-0 through office-4, and Avg.The passage supplies these labels without specifying their table layout or values.Source: C. Additional Experimental Results on… · p. 12

NICE-SLAM

• Table A reports scene-level reconstruction results across 8 indoor Replica scenes.The caption notes that ∗ marks baselines re-trained with ground-truth pose.Source: vMAP · p. 12
• Novel-view synthesis was evaluated on newly generated sequences by comparing rendered views with ground-truth views.The reported metrics are depth L1 error, PSNR, SSIM, and LPIPS.Source: vMAP · p. 12
• Table C presents 2D novel-view synthesis rendering results on the Replica dataset.Source: Table C · p. 13

D. Visualisation of Object-level Hole-filling

• vMAP shows significantly better hole-filling in unobserved regions than iMAP and NICE-SLAM, with visual consistency.The authors attribute this to the disentangled object representation design.Source: D. Visualisation of Object-level… · p. 13
• The Replica visualization presents 2D novel-view synthesis renderings.The caption notes that the results are better viewed zoomed in.Source: Figure C · p. 14
• The hole-filling visualization depicts object-level reconstruction.Source: Figure D · p. 15