Source-linked AI summary
Inverse Rendering for Modeling with Line Primitives
Kenji Tojo, Ariel Shamir, Nobuyuki Umetani, Bernd Bickel
TL;DR
Fuzzy and anisotropic structures are difficult to represent with explicit surfaces, while volumetric methods lack low-dimensional structure for standard graphics pipelines. The paper reconstructs them as anti-aliased explicit line primitives optimized by stochastic differentiable rasterization. It outperforms surface-based approaches on fuzzy boundaries, matches volumetric quality, and supports explicit-geometry workflows.
Problem
Surface representations struggle with fuzzy, anisotropic geometry, while implicit volumetric representations are difficult to integrate with standard rasterization, geometric reasoning, and physical simulation.
Method
The method reconstructs 3D line segments and polylines using subpixel anti-aliased rasterization, stochastic gradients, and periodic topology updates.
Results
The method outperforms surface-based approaches for complex anisotropic boundaries and achieves performance comparable to volumetric methods while using explicit geometry.
Takeaways & Limitations
The explicit line representation supports standard graphics pipelines, cross-platform deployment, advanced shading, physics-based animation, and interactive rendering.
Takeaways & Limitations
Complete topology, microscale fiber orientations, and connectivity remain difficult for heavily occluded or barely visible structures.
Abstract
from arXiv · showhide
Faithfully capturing diverse real-world objects with fuzzy, anisotropic structures, such as hair, fur, fibers, and textiles, for efficient real-time visualization remains challenging. Recent radiance field reconstruction methods capture these structures from multi-view images using translucent volumetric primitives such as 3D Gaussians rather than opaque low-dimensional primitives (e.g., triangles, line segments, and polylines), thereby limiting compatibility with standard depth-tested rasterization, reflection modeling, and physical simulation. We present an inverse rendering method for reconstructing fuzzy geometry using explicit line segments, which are rasterized on a subpixel grid for anti-aliasing to reproduce a semi-transparent appearance. While straightforward to render, optimizing numerous line primitives to match target images poses a significant challenge. We address this by introducing a stochastic differentiable rasterizer for line segments that produces informative gradients with respect to vertex positions, attributes, and discrete connectivity. Experiments on synthetic and real-world datasets show that our method outperforms surface-based approaches in capturing fuzzy boundaries and achieves quality comparable to volumetric representations while relying entirely on explicit geometry. The resulting representation integrates seamlessly with standard graphics pipelines, enabling cross-platform rendering, various shading models, and physical simulation.
1 Introduction
The paper targets fuzzy, anisotropic structures that challenge surface representations, proposing explicit line primitives with anti-aliased rasterization and differentiable optimization to preserve fidelity and graphics-pipeline compatibility.
- Surface representations struggle with fuzzy structures whose ambiguous boundaries and appearance emerge from many thin elements.
- Volumetric methods capture fine-scale appearance but are difficult to integrate with depth-tested rasterization, geometric reasoning, and physical simulation.
- The method models fuzzy geometry as explicit 3D vertices connected into line segments or polylines, rasterized on a subpixel grid with MSAA.
- Aggregating many opaque line segments produces semi-transparent appearance while preserving tangent directions and polyline connectivity.
- A stochastic differentiable rasterizer supplies gradients for vertex positions, attributes, and discrete connectivity during line-primitive optimization.
- Experiments report stronger fuzzy-boundary capture than surface methods and quality comparable to volumetric approaches using entirely explicit geometry.
2 Related Work
Prior work captures appearance with differentiable rendering, radiance fields, volumetric splats, and curve or surface proxies, but generally lacks adaptable explicit structure for dense fuzzy geometry.
- Differentiable rendering methods span light transport, rasterization, vector graphics, parametric geometry, and programmable shaders.
- Existing curve methods commonly assume fixed topology and use analytic intersections, sequential segments, camera-facing quads, or tubular meshes.
- Textured surface proxies remain difficult for dense microstructures because coarse geometry does not match their underlying directional structures.
- NeRFs and 3D Gaussian Splatting represent complex appearance volumetrically, while 3DGS lacks explicit connectivity for geometry-processing algorithms.
- The representation supports hair reflectance rendering and mass-spring animation through its explicit strand structure.
- The paper reconstructs subpixel-width line segments whose fuzzy appearance emerges through screen-space filtering, yielding an explicit 1D manifold.
- Unlike sparse stylized 1D abstractions, the approach optimizes dense line collections to reproduce rich real-world appearance.
3 Preliminary
The preliminary framework explains stochastic differentiation for opaque primitives under z-buffer visibility, enabling gradients through discrete fragment selection without explicit sorting.
- The stochastic rasterizer extends discrete opaque-primitive differentiation to line segments with anti-aliasing and adaptive 1D topology.
- Opaque primitives use z-buffering to select the frontmost fragment rather than back-to-front alpha blending.
- DiffSoup assigns continuous opacity during optimization, stochastically activating fragments with probability equal to their interpolated opacity.
- The loss averages L1 color error over stochastic renderings.
- Likelihood-ratio gradients combine standard color gradients with a score-function term for differentiating fragment existence.
- The log-probability gradient can be evaluated without sorting fragments, preserving efficient depth-tested rasterization.
- At test time, the fragment opacity threshold is set to 0.5.
4 Method
The method reconstructs fuzzy, anisotropic structures as explicit line primitives and optimizes them through stochastic differentiable rasterization. Subpixel rasterization, anti-aliasing, opacity gradients, and adaptive connectivity updates support detailed reconstruction from unstructured initialization.
- 4.1 Scene Representation with Line Primitives: The scene is represented by 3D vertices connected into non-branching polylines, with explicit geometry compatible with depth testing and geometry processing.Vertices may carry learnable features or tangents, while edges define line segments through vertex-index pairs.
- 4.1 Scene Representation with Line Primitives: Per-vertex attributes and per-edge existence probabilities model view-dependent appearance and allow primitives to emerge or disappear during optimization.Each edge opacity α_e ∈ [0, 1] represents its existence probability.
- 4.2 Differentiable Rasterization of Line Primitives: Lines are rasterized with Bresenham coverage on a 2× subpixel grid, then filtered to produce anti-aliased pixel colors without explicitly optimizing line width.This provides consistent subpixel coverage for dense arrangements of thin fibers and avoids instability from continuous width parameters.
- 4.2 Differentiable Rasterization of Line Primitives: The differentiable rasterizer propagates positional gradients through microedges induced by line motion and computes stochastic fragments for depth-tested rendering.Endpoint colors and depths are interpolated per fragment, and the stochastic depth test produces the rendered image.
- 4.2 Differentiable Rasterization of Line Primitives: Opacity gradients approximate joint fragment probabilities with independent per-fragment probabilities and evaluate gradients only over the four fragments inside the loss pixel.The forward path uses a 6 × 6 Gaussian filter, whereas the backward path uses a 2 × 2 box-filter neighborhood.
5 Results
Across synthetic and real-world evaluations, the method reconstructs fuzzy geometry with explicit line primitives while achieving strong perceptual quality, interactive rendering, and compatibility with downstream graphics workflows. Its main weakness is reduced performance on large, rounded, visually flat regions where triangles cover surfaces more efficiently.
- Rendering performance: Even with 4× more pixels for full-quality rendering, the method remains faster than 3DGS while achieving better perceptual quality.Hardware MSAA or disabled anti-aliasing provides explicit quality–performance trade-offs.
- Synthetic dataset: On the Shelly dataset, our method outperforms surface-based approaches and achieves perceptual quality comparable to volumetric methods.It performs particularly well on complex anisotropic boundaries and obtains substantially better SSIM and LPIPS than volumetric methods, despite similar or slightly lower PSNR.
- Synthetic dataset: Compared with DiffSoup and VolSurfs, the method better preserves thin fuzzy boundaries and fine geometric details, while matching 3DGS in overall visual quality.It also captures connected curve structures more faithfully than 3DGS.
- Practical comparison: Compared with artist-authored Shelly assets requiring hours to days of specialized work, the method reconstructs similarly complex structures from multi-view images within one hour.This advantage is reported to be even more apparent for challenging real-world captures.
- Real-world captures: The method reconstructs detailed textures, fuzzy boundaries, plausible outlines, and directional fiber structures across real-world objects with varied fuzziness and spatial complexity.The reconstructed geometry remains coherent despite occlusions and defocus blur, and can be viewed interactively on a laptop.
- Applications: Explicit line primitives support hair reflectance rendering, GPU mass-spring animation, offline ray-traced cylinders, and integration with surface-based scenes.A simulation processes 2M vertices in approximately 55 ms/frame, although millions of fibers remain challenging to handle in real time.
- Ablations: The method outperforms all baseline ablations across reported metrics and scenes except the Woolly scene reconstructed using only triangle primitives.The Gaussian reconstruction filter improves fuzzy boundaries and directional-structure preservation over a box filter.
- Failure cases: The Woolly scene exposes a limitation: its large spatial extent and rounded shell shape are difficult to cover efficiently with line primitives.Triangle primitives produce sharper contrast in textured regions and smoother flat regions, motivating a hybrid representation.
6 Limitations and Future Work
The method has several limitations involving occlusion, microscale structure recovery, opacity optimization, line-width estimation, and viewing-distance adaptation.
- Complete topology, especially heavily occluded hair interiors, likely requires stronger priors or richer capture setups.
- Barely visible structures may lack fully accurate microscale fiber orientations and connectivity.
- The method leaves learned connectivity updates and more principled variance reduction for future work.
- Recovering line width from subpixel observations remains fundamentally challenging.
- Viewing-distance adaptation may require level-of-detail techniques or a hybrid triangle–line representation.
Supplemental Material: Inverse Rendering for Modeling with Line Primitives
The supplemental material adds details about the real-world capture dataset and optimization settings.
- The supplemental material provides additional details on the real-world capture dataset.
- The supplemental material provides additional details on optimization settings.
- Dataset capture and optimization are the stated supplemental-material topics.
A Additional Capture Details
The Fuzzy dataset uses COLMAP camera poses and NeuS2 coarse-surface reconstruction to initialize line primitives, while noisy background fields can clutter extracted surfaces.
- COLMAP-estimated camera poses support coarse-surface reconstruction for the Fuzzy dataset.
- NeuS2 recovers a coarse object outline in approximately three minutes for line-primitive initialization.
- Low-texture black backgrounds can produce noisy signed distance fields and cluttered marching-cubes surfaces.
B Additional Optimization Details
The supplemental experiments use distance-band initialization, scheduled optimization hyperparameters, matched baseline initialization, and documented dataset and proxy-mesh statistics.
- Initialization: Seed points are uniformly sampled within an experimentally chosen distance band of 0.1 to avoid missing fine fuzzy structures.
- Hyperparameters and scheduling: Positional learning rates are reduced relative to common 3DGS settings and exponentially decayed during optimization.The initial rates are 2.5×10^-5 for Shelly and 1.6×10^-5 for the Fuzzy dataset, decayed to one-fifth.
- Dataset statistics: Source images are captured at 6720×4480 px and optimized after 4× downsampling of undistorted images.
- Dataset statistics: Table A2 reports shortest-axis, longest-axis, and diagonal extents for Shelly coarse proxy meshes.
- Hyperparameters and scheduling: The length-regularization weight L_len is set to 1.0 × 10^-3.
- Baseline methods: The 3DGS baseline uses the same coarse proxy surface and distance-band sampling, with 500K Gaussian primitives versus 2M line vertices.