Source-linked AI summary
MEGA: Object-Level Mesh Extraction from 3D Gaussian Splatting via Spatial Visual Distillation
Liwei Liao, Yingkui Zhang, Qianqian Tong, Ronggang Wang
TL;DR
3DGS lacks coherent object-level geometry, and existing mesh extraction often produces scene-level or non-watertight surfaces. MEGA uses segmented 3DGS views to train neural surface reconstruction; it reports state-of-the-art performance among meshable methods and competitive results on LERF-Mask, with slight mesh deviations that may cause minor penetration during interactions.
Problem
3DGS lacks explicit object-level geometry, while existing mesh methods often focus on scene-level extraction and produce non-watertight surfaces.
Source: §1 · p. 1, §2.2 · p. 2Method
MEGA segments a target object, samples multi-view camera observations from 3DGS, and distills them into a neural surface representation for mesh extraction.
Source: §1 · p. 1, §3.2 · p. 4, §3.3 · p. 4Results
MEGA achieves state-of-the-art performance among meshable methods and competitive performance on LERF-Mask compared with its SAGO teacher.
Source: §4.2 · p. 5Takeaways & Limitations
MEGA provides object-level meshes for physical interaction alongside 3DGS representations for visual rendering.
Source: §1 · p. 1Imperfect alignment between Gaussian fitting and object surfaces may cause slight mesh deviations and minor penetration during interactions.
Source: §5 · p. 7
Abstract
from arXiv · showhide
Mesh extraction from 3D Gaussian Splatting (3DGS) aims to endow 3D Gaussians with accurate geometric structures, enabling explicit and precise 3D occupancy. However, existing methods primarily focus on scene-level mesh extraction, making them unable to represent object-level occupancy and often resulting in non-watertight surfaces. To overcome these limitations, we propose \textbf{MEGA} (\underline{M}esh \underline{E}xtraction from \underline{GA}ussians), a ``segment-then-mesh'' framework for extracting object-level, watertight meshes from complex 3DGS scenes. At the core of MEGA are \textbf{Spatial Visual Distillation (SVD)} and a mask-guided neural surface reconstruction module. SVD treats the 3DGS model as a teacher, sampling diverse camera poses and rendering the corresponding views of each segmented object. These observations are then used to train a mesh reconstruction model through photometric supervision. Extensive experiments on several widely used benchmarks demonstrate that MEGA achieves state-of-the-art performance in recovering accurate object-level 3D occupancy. Moreover, MEGA enables complex physical interactions by combining high-quality object-level meshes for geometric occupancy with 3DGS representations for photorealistic rendering.
1. Introduction
3DGS represents visual appearance but lacks coherent object-level geometry for reliable physical interaction, while existing mesh methods often produce scene-level or non-watertight surfaces. MEGA extracts object-level meshes from 3DGS and uses Spatial Visual Distillation to transfer multi-view visual knowledge into a neural reconstruction model.
Problem
- Existing 3DGS mesh methods often produce scene-level meshes unsuited to object interaction, and non-watertight surfaces that cannot provide complete, reliable occupancy.3DGS itself lacks coherent surface geometry for reliable physical interaction.Source: §1 · p. 1
Motivation
- Object-level meshes provide geometry for independently localizing, manipulating, grasping, and simulating objects in photorealistic 3DGS scenes.They supply an interface between visual representations and physically interactive environments.Source: §1 · p. 1
MEGA
- MEGA extracts watertight object-level meshes aligned with the underlying 3D Gaussians, providing explicit occupancy for interaction and manipulation.The framework uses Spatial Visual Distillation to render segmented objects from diverse virtual-camera viewpoints and train a neural mesh reconstruction module.Source: §1 · p. 1
Contributions
- Spatial Visual Distillation transfers multi-view visual knowledge from 3DGS into a neural mesh reconstruction model.The paper presents MEGA as an early systematic exploration and a baseline for physically interactive 3DGS representations.Source: §1 · p. 1
2. RELATED WORKS
Prior 3DGS segmentation methods trade per-scene optimization against inference-time operation, while existing mesh extraction methods often lack object-level, watertight geometry. MEGA adopts online segmentation and targets object-level watertight meshes.
2.1. 3DGS Segmentation
- Offline segmentation methods such as SAGA, Click-Gaussian, and COB-GS require per-scene optimization that typically takes tens of minutes; online methods avoid per-scene training, and MEGA adopts SAGO.SAGO is described as enabling interactive 3D segmentation with sub-second latency.Source: §2.1 · p. 2
2.2. Mesh Extraction from 3DGS
- Mesh extraction methods include surface-alignment regularization, convex collision hulls, 2D Gaussian disks with TSDF fusion, and opaque-triangle reconstruction, but most target scene-level meshes.SAGD's convex polyhedra are too coarse for complex object geometry, while extracted meshes are frequently non-watertight.Source: §2.2 · p. 2
- MEGA addresses these limitations with object-level, watertight, coordinate-aligned meshes for explicit occupancy in complex 3DGS scenes.Unlike convex-hull collision meshes, its stated goal is accurate object geometry.Source: §2.2 · p. 2
3. Method
MEGA segments a target object from 3DGS, distills rendered multi-view observations into a neural surface, then extracts an aligned watertight mesh.
3.1. Overview
- MEGA follows a segment-then-mesh pipeline: it obtains an object-level Gaussian subset, distills multi-view information into a neural surface, and extracts a watertight mesh.It can optionally refine boundary artifacts in the object-level Gaussian representation.Source: §3.1 · p. 3
3.2. Preliminaries: Interactive 3DGS Segmentation
- MEGA adopts SAGO for online object segmentation, using virtual drones, SAM2 trackers, and Mask-shaped Frustum Filtering to provide an initial spatial prior.SAGO’s next-best-view objective selects a view maximizing the predicted foreground update while preserving foreground consistency.Source: §3.2 · pp. 3–4
3.3. Spatial Visual Distillation
- SVD samples virtual cameras around the segmented object and renders RGB views with propagated masks as teacher observations for neural surface distillation.Hemispherical sampling supplies diverse observations of the object’s appearance and geometry.Source: §3.3 · p. 4
- A neural signed distance field is optimized to match teacher views, with mask supervision and Eikonal regularization.The objective combines photometric loss with Eikonal regularization.Source: §3.3 · p. 4
- After optimization converges, evaluating the learned SDF on a dense volumetric grid and applying marching cubes extracts an explicit triangular mesh, spatially aligned with Θobj and providing watertight object-level 3D occupancy.Source: §3.3 · p. 4
4. EXPERIMENTS
MEGA is evaluated across silhouette-matching benchmarks, mesh-extraction comparisons, applications, and ablations. Results highlight its object-level meshes and the effects of sampling density, neural surface distillation, and boundary trimming.
4.1. Experimental Setting
- Evaluations project extracted meshes onto test views and compare silhouette mIoU and mAcc against masks across LERF-mask, NVOS, and SPIn-NeRF, using 216 virtual viewpoints.The mesh-extraction baselines include SAM3D, PGSR, MeshSplatting, and 2DGS.Source: §4.1 · p. 5
4.2. Quantitative Evaluation
- MEGA achieves state-of-the-art performance among meshable methods, SVD improves existing mesh extractors as a plug-in, and MEGA is competitive with teacher SAGO on LERF-Mask mIoU.SAGO serves as MEGA’s teacher model in the LERF-Mask comparison.Source: Table 2 · p. 5, §4.2 · p. 5
4.3. Qualitative Comparison
- Figure 4 contrasts MEGA’s accurate, watertight, well-aligned meshes with SAM3D’s geometric deviations and the bottom-surface holes in PGSR, MeshSplatting, and 2DGS.The non-watertight meshes may lead to severe penetration in physical simulations.Source: Figure 4 · p. 6, §4.3 · p. 6, Figure 6 · p. 7
4.4. Applications
- MEGA’s joint Gaussian-mesh representation supports human–object interaction and physics simulation, including operation of mesh-only physical interaction engines on 3DGS objects.The paper presents four examples: two for human–object interaction and two for physics simulation.Source: §4.4 · p. 6, Figure 5 · p. 7
4.5. Ablation Study
- Once sampling density reaches a level such as 3×72, further increases do not necessarily yield significant mIoU gains on LERF-Mask.The ablation varies horizontal and vertical sampling densities separately.Source: §4.5 · pp. 6–7, Figure 7 · p. 7
- SVD-augmented 2DGS and PGSR outperform their independently trained counterparts but remain below full MEGA on mIoU and mAcc.This ablation supports SVD’s use as a plug-in module for existing mesh-extraction methods.Source: §4.5 · p. 7
- Boundary trimming does not change mesh extraction results, but it improves the visual quality of the Gaussian-mesh representation by trimming rough 3DGS object surfaces.Figure 7(c) illustrates the trimming.Source: §4.5 · p. 7, Figure 7 · p. 7
5. CONCLUSION
MEGA uses Spatial Visual Distillation to supervise neural surface reconstruction from multi-view observations of segmented 3DGS. It extracts watertight, geometry-aligned object meshes, although imperfect Gaussian-surface alignment can still allow minor penetration during interactions.
5. CONCLUSION
- MEGA uses segmented 3DGS as a teacher to generate multi-view camera pairs that supervise a neural surface reconstruction network.The approach is framed as Spatial Visual Distillation.Source: §5 · p. 7
- The reconstruction produces watertight, geometry-aligned object-level meshes for explicit 3D occupancy.Source: §5 · p. 7
- Imperfect alignment between Gaussian fitting and object surfaces can leave slight deviations and permit minor mesh penetration during interactions.The paper identifies further 3DGS optimization using better-aligned meshes as future work.Source: §5 · p. 7