Volume Encoding Gaussians: Transfer Function-Agnostic 3D Gaussians for Volume Rendering

• Large HPC datasets can exceed end-user GPU memory, while visualization may require HPC resources, limiting accessibility and interactivity.Source: §1 · p. 1
• Base 3DGS embeds view-dependent color and opacity, limiting interactive transfer-function changes; prior editable-Gaussian methods struggle on unseen transfer functions and grow in cost as models are composed.Source: §1 · p. 1
• VEG stores scalar values rather than per-Gaussian color and opacity, separating the data representation from visual appearance for dynamic arbitrary mappings.Source: §1 · p. 1
• Opacity-guided training represents the full volume function range without separately training and composing models for multiple transfer functions.Source: §1 · p. 1

2 RELATED WORK

• The section situates volume visualization amid growing dataset sizes and reviews the field through a survey, then outlines neural, image-based, and 3DGS work.Source: §2 · p. 2

2.1 Neural Representations for Volume Data

• Neural representations model scalar values from positions and can provide compressed volume rendering, while image-based methods can fit simulation workflows, support varied data types, and operate independently of dataset size.Source: §2.1 · p. 2

2.2 Image-Based Methods for Volume Visualization

• Prior image-based methods include interactive transfer-function exploration; iVR-GS edits palette colors after training, whereas VEG learns the scalar field directly to avoid training multiple models per transfer function.Source: §2.2 · p. 2

2.3 3D Gaussian Splatting

• 3DGS offers explicit, fast rendering, but its view-dependent color and opacity and scene-reconstruction initialization do not fit scientific volume workflows requiring interactive transfer functions.Source: §2.3 · p. 2
• VEG adopts Blinn-Phong lighting and vector quantization, which clusters model parameters into a codebook to reduce storage while maintaining reconstruction quality.Source: §2.3 · pp. 2–3

3.1 Volume Types

• Structured volumes place scalar values on regular grids, while unstructured volumes explicitly store geometry and require cell lookup before interpolation.Source: §3.1 · p. 3

3.2 The VEG Representation

• VEG uses independently positioned, scaled, and rotated ellipsoidal points to encode geometry and scalar values, including unstructured volumes without storing connectivity.Source: §3.2 · p. 3

3.3 Rendering of VEG

• The rendering target is emission-absorption raymarching, approximated by splatting Gaussians and alpha-blending them in front-to-back order.Source: §3.3 · p. 3
• Applying a transfer function before accumulation maps each Gaussian to color and opacity and allows low-opacity Gaussians to be culled.Source: §3.3 · p. 3
• A trainable Gaussian weight multiplies transfer-function opacity, restoring direct opacity optimization during training.Source: §3.3 · p. 3
• VEG attaches Blinn-Phong lighting parameters to each Gaussian and backpropagates gradients through lighting and transfer-function rendering to optimize scalar values.Source: §3.3 · p. 3

4.1 Initialization

• Initialization converts scalar-valued volume vertices into Gaussians and sets their sizes by neighbor proximity, with larger Gaussians in sparse regions.The method drops mesh connectivity; lower-density areas receive larger initial Gaussians.Source: §4.1 · p. 3, Figure 2 · p. 4
• Randomly dropping most data points before initialization creates compact models and reduces the data that must be stored for later training.The paper also reports that random initialization can perform well when point sampling is unavailable.Source: §4.1 · pp. 3–4

4.2 Training

• Differentiable image-space training renders VEG under sampled camera and transfer-function settings, compares outputs with reference images, and learns volume geometry and scalar values.The image loss combines L1 and SSIM; training also optimizes normals and lighting attributes.Source: §4.2 · p. 4
• Multiple opacity maps weight scalar-range regions more equally and provide gradients for adjusting scalar values; Fig. 3 shows more equal Gaussian density than linear or inverse maps.The maps use linear drop-off around centers distributed across the function range.Source: §4.2 · p. 4, Figure 3 · p. 5
• Linear opacity training concentrates Gaussians in high-value regions, inverse linear training favors low-value regions, and three opacity maps produce equal density across both.Source: Figure 3 · p. 5
• Adaptive densification adds Gaussians where positional gradients indicate poor reconstruction, while pruning removes low-weight Gaussians to control model density.The authors report these heuristics produce compact models suited to the reconstructed volume's complexity.Source: §4.2 · pp. 4–5

5 EVALUATION

• The interactive renderer replaces per-basic-set color and opacity sliders with colormap selection and an opacity-map creation tool.A WebGPU web application port illustrates the method’s potential for low-memory client devices.Source: Figure 4 · p. 5, §5 · p. 5

5.1 Setup

• Across two structured and two unstructured volumes, VEG and composed iVR-GS were tested on novel views under unseen colormaps and broad or narrow opacity maps.The test set used five colormaps unseen during training and opacity maps designed to represent volume-like and surface-like renderings.Source: §5.1 · p. 5, §5.2 · p. 6

5.3 Ablation Study

• Randomly initialized VEG performed about as well as the default except on training TFs, where the default was better.The authors attribute robustness to densification and pruning, which can create and remove geometry during training.Source: §5.3 · p. 8
• Removing per-Gaussian weights significantly reduced reconstruction quality and left no weight-threshold pruning, producing 563,223 Gaussians versus 49,969 by default.The weight is multiplied by transfer-function-determined opacity during rendering and also supports pruning ineffective Gaussians.Source: §5.3 · p. 8
• A single linear opacity map performed poorly on other opacity tests; more training maps appeared to improve unseen-map generalization, while training time and Gaussian count fell as map count rose.The authors suggest wider maps can leave overlapping scalar regions without learned features, while narrower maps yield sparser training images.Source: §5.3 · pp. 8–9
• Adding 2× and 3× zoom views did not improve zoomed rendering and worsened default-view quality, despite using more than twice the default model’s Gaussians.The default model retained near-equal quality on zoomed views; Table 4 compares the default and zoom-augmented training sets.Source: Table 4 · p. 9, §5.3 · p. 9

5.4 Complex TF Showcase

• On a head CT volume, VEG produced high-quality novel views for four transfer functions with unseen color and opacity maps, though custom maps showed more errors, especially overfilled semi-transparent areas.The examples include multiple narrow opacity bumps and a semi-transparent bump paired with a high-opacity bump.Source: §5.4 · p. 9, Figure 7 · p. 9

6 CONCLUSION

• VEG directly encodes scalar-valued fields rather than color and opacity, decoupling volume data representation from visual appearance.The approach is tailored to structured and unstructured scientific volume rendering.Source: §6 · p. 9
• VEG achieves high reconstruction quality with robust generalization to unseen transfer functions, using less than 5 megabytes for every studied dataset.Source: §6 · p. 9
• Future work targets time-varying volumes, distributed training for very large datasets and HPC contexts, and direct volume reconstruction rather than image rendering.The paper also proposes compact VEG models for volume sampling.Source: §6 · p. 9