Source-linked AI summary
SILSA: Sliding-Window Slice Latents for Topology-Preserving High-Resolution 3D Generation
Tianjiao Yu, Xinzhuo Li, Yifan Shen, Ying Shen, Kiet A. Nguyen, Adheesh Sunil Juvekar, Ismini Lourentzou
TL;DR
High-resolution 3D generators can fragment surfaces and weaken topological consistency, while compact representations may lose local geometric correspondence. SILSA generates shapes from fixed, overlapping multi-axis slice latents, with topology-aware reconstruction and cross-axis coordination. It improves image-to-3D fidelity while reducing token use, training memory, and inference time.
Problem
Method
SILSA uses fixed overlapping slice latents along three axes, a SliceVAE with slice-level topology supervision, and a Volumetric Anchor Lattice in a single-stage rectified-flow generator.
Source: §1 · pp. 2–3Results
Takeaways & Limitations
Across reconstruction, generation, and efficiency evaluations, SILSA improves geometric fidelity, preserves thin and highly connected structures, and reduces token, memory, and inference costs.
Source: §5 · p. 9SILSA’s fidelity depends on training-distribution diversity and scale and input-image quality; out-of-distribution structural patterns or ambiguous, low-information views may reduce fidelity.
Source: F Discussion · p. 21
Abstract
from arXiv · showhide
High-resolution 3D generation increasingly relies on voxel latents and multi-stage pipelines that first predict active structure and then synthesize local geometry. While effective, this design fragments continuous surfaces into many local tokens, inflates generation cost, and often weakens topological consistency for thin or highly connected shapes. We introduce SILSA, a topology-aware 3D generation framework that represents shapes with compact sliding-window slice latents. Instead of generating expensive voxel tokens, SILSA uses a fixed set of overlapping slices along the three canonical axes, where each token summarizes a local depth window to preserve cross-sectional continuity and support single-stage rectified-flow generation. A Slice VAE encodes oriented surface samples into multi-axis slice latents and reconstructs them with a sparse volumetric decoder, while a Volumetric Anchor Lattice coordinates directional slice streams through a shared 3D workspace. To preserve structural correctness, we introduce slice-level topology supervision that matches persistence diagrams and aligns Betti transitions across neighboring slices. Experiments show that SILSA improves structural fidelity while substantially reducing generation cost. SILSA improves PSNR by $8.7\%$, coverage by $5.96$ absolute points, and Betti error by $9.2\%$ over the strongest baseline, while using $70.0\%$ fewer tokens than the next-most compact baseline and over $98\%$ fewer tokens than sparse or hierarchical tokenizers, effectively reducing training memory by $40.4\%$ and inference time by $58.5\%$. Qualitative results further show improved preservation of thin structures, repeated components, and long-range connectivity.
1 Introduction
SILSA addresses structural failures in high-resolution 3D generation with fixed, spatially grounded slice latents and topology-aware training. Its image-to-3D results improve fidelity and efficiency while preserving fine structures.
1 Introduction
- Existing high-resolution generators can match overall appearance yet break thin parts, openings, and repeated components, altering connectivity and part relationships.Dense voxels scale cubically with resolution, while sparse or hierarchical token counts can grow with thin supports, repeated parts, or complex topology; compact alternatives weaken local geometric correspondence.Source: §1 · p. 2
- SILSA represents shapes as overlapping sliding-window slice latents along three axes, using 3N tokens regardless of occupancy, surface area, or part complexity.Each token summarizes a local depth window, and the three axis-wise sequences provide complementary cross-sectional views.Source: §1 · pp. 2–3
- A SliceVAE, shared Volumetric Anchor Lattice, and slice-level topology losses combine geometric reconstruction with cross-axis coordination and topology supervision.The losses match persistent-homology structure and align Betti transitions across adjacent slices.Source: §1 · pp. 2–3
- SILSA improves image-to-3D PSNR by 8.7% and coverage by 5.96 absolute points over the strongest baselines, while using 384 tokens and reducing memory and inference time.The SliceVAE reduces Betti error by 9.2% relative to the strongest reconstruction baseline; token use is 70.0% lower than the next-most compact baseline, with training memory down 40.4% and inference time down 58.5%.Source: §1 · p. 2
2 Related Work
Prior 3D representations trade efficiency against spatial structure, while cross-sectional slices expose components, holes, and connectivity changes. SILSA uses multi-axis slices as fixed sliding-window generative latents and supervises topology within and between neighboring slices.
2 Related Work
- Dense voxels provide spatial grounding but scale cubically; compact triplanes and set latents weaken local correspondence, while sparse or hierarchical tokenizers use data-dependent counts and often require multi-stage generation.Cross-sectional representations expose components, holes, and connectivity changes; SILSA uses multi-axis cross-sections as learned generative latents with fixed sliding-window tokens.Source: §2 · p. 3
- SILSA’s topology supervision matches persistence within slices and Betti transitions across neighboring slices, avoiding expensive full-volume topology matching.The approach builds on prior persistent-homology and Betti-based losses for preserving connectivity and holes.Source: §2 · p. 3
3 Method
SILSA replaces voxel-level tokens with spatially grounded, overlapping slice latents and a single-stage rectified-flow generator. Its SliceVAE reconstructs geometry while topology losses and shared volumetric memory support structural and cross-axis consistency.
3 Method
- SILSA generates compact slice latents instead of voxel tokens, avoiding a separate active-voxel prediction stage while addressing fragmented surfaces and costly generation.A shared Volumetric Anchor Lattice coordinates slice streams from the three axes during denoising.Source: §3 · p. 3
3.1 Topology-Aware Slice VAE
- Overlapping sliding windows encode local geometry into 3N axis-aligned slice tokens, which are fused in a shared volumetric grid and decoded through sparse upsampling.The decoder progressively refines and prunes active cells before extracting a mesh.Source: §3.1 · pp. 3–5
3.2 Slice-Wise Topology-Preserving Loss
- Slice-wise topology supervision matches persistence diagrams for components and holes, and aligns Betti transitions to track where topology changes across neighboring slices.The two terms address within-slice topology and its evolution along each canonical axis.Source: §3.2 · pp. 5–6
3.3 Rectified Flow Generation with Volumetric Anchors
- A rectified-flow transformer generates image-conditioned slice latents, while the Volumetric Anchor Lattice lets tokens from different axes share spatial evidence during denoising.Each token reads from and writes to the anchor plane for its axis and depth position.Source: §3.3 · p. 6
4 Experiments
SILSA improves image-to-3D generation and VAE reconstruction quality while reducing token and compute costs; ablations support cross-axis coordination and topology supervision.
Experiment Setup
- Evaluation uses 200 randomly sampled Toys4K assets and 50 in-the-wild images with no training-set overlap; Trellis-500K is used for training.The setup compares VAE reconstruction with CD, F-score, and Betti Error, and image-to-3D generation with alignment, quality, fidelity, and distributional metrics.Source: §4 · p. 7
Image-to-3D
- PSNR rises from 30.12 to 32.74 over SparseFlex (8.7% relative gain), and coverage rises from 73.12% to 79.08% (5.96-point absolute gain).SILSA also achieves the best FD, PSNR, coverage, and MMD, while matching the best KD and LPIPS; SparseFlex has a slightly higher CLIP score.Source: §4 · p. 7
VAE Reconstruction Evaluation
- Betti-Err falls from 1.743 to 1.582 versus SparseFlex, a 9.2% relative reduction, alongside improvements in CD, F-Scores, and IoU.Table 2 covers geometric, volumetric, and topology-aware reconstruction metrics; Betti-Err measures mismatch in connected components and holes.Source: §4 · p. 7, Table 2 · p. 8
Efficiency analysis
- 384 tokens are over 98% fewer than Trellis, XCube, and SparseFlex; inference time falls from 0.82s/shape to 0.34s/shape.Relative to Dora, training memory falls from 14.6GB to 8.7GB, and training time falls from 0.38s/iter to 0.21s/iter; SILSA uses a one-stage generator.Source: §4 · pp. 7–8, Table 3 · p. 8
Ablations
- VAL achieves the lowest CD (0.59) and Betti-Err (1.58); D=16 gives the best resolution trade-off, and the full topology loss lowers Betti-Err from 4.43 to 1.58.Independent slice streams perform substantially worse, while direct cross-attention improves consistency; D=32 slightly improves IoU but worsens CD and Betti-Err.Source: §4 · pp. 8–9, Table 4 · p. 8
5 Conclusion
SILSA represents shapes with compact multi-axis slice latents, combining topology-supervised reconstruction with a single-stage generator coordinated by a Volumetric Anchor Lattice. Evaluations report improved fidelity and preservation of thin or highly connected structures alongside lower token, memory, and inference costs.
Conclusion
- SILSA uses compact sliding-window slice latents along three canonical axes, with a SliceVAE, topology supervision, and a Volumetric Anchor Lattice in a single-stage rectified-flow generator.Across reconstruction, generation, and efficiency evaluations, it improves geometric fidelity and preserves thin and highly connected structures while reducing token count, memory, and inference cost.Source: §5 · p. 9
A Related Work
Related work spans 3D generation pipelines, compact shape representations, and topology-aware learning. SILSA is positioned as a multi-axis slice-latent approach with slice-based topology supervision.
3D Generation
- 3D generation progresses from lifted 2D priors and multi-view reconstruction to native generators over learned latent spaces; SILSA instead generates canonical multi-axis slices.The cited approaches include score distillation [52], multi-view systems [44] [60], and native 3D generators [89].Source: A Related Work · p. 16
Latent Representations for 3D Shapes
- Shape representations trade spatial structure against efficiency: voxels scale cubically, while compact latents can weaken local geometric correspondence and sparse or hierarchical approaches preserve locality.SILSA uses a compact 3N-token layout of learned, axis-aligned slices, combining spatial grounding with a bounded token count.Source: A Related Work · p. 16
Topology-Aware Learning
- Topology-aware learning applies persistent-homology, Betti-based, and skeleton-based objectives, but repeated volumetric persistent-homology supervision is expensive in high-resolution generative training.SILSA matches persistence diagrams within individual slices and Betti transitions across neighboring slices.Source: A Related Work · p. 16
B Illustration of Multi-Axis Slice Topology
The illustration tracks connected components and holes as cross-sections move through a shape along the three canonical axes. Its topology losses supervise both per-slice structure and the locations of changes across adjacent slices.
Multi-Axis Slice Topology
- Each slicing direction yields a depth-ordered sequence of cross-sections, with β0 components and β1 holes tracked over slice depth.Figure 5 depicts blue intervals for connected components and red intervals for holes; interval endpoints indicate topological transitions.Source: B Illustration of Multi-Axis Slice… · p. 16, Figure 5 · p. 17
Topology Supervision
- Per-slice persistence matching supervises topology within each cross-section, while Betti-transition matching aligns where topological changes occur across neighboring slices.Transitions include components or holes appearing or disappearing, and regions merging or splitting.Source: B Illustration of Multi-Axis Slice… · p. 16, Figure 5 · p. 17
Open-Surface Shapes
- Table 5 reports VAE reconstruction on open-surface shapes.Source: Table 5 · p. 17
C Implementation Details
SILSA uses a three-axis slice representation and sparse volumetric decoding, trained on Trellis-500K with separate VAE and rectified-flow training stages.
C Implementation Details
- Training uses Trellis-500K meshes normalized to unit boxes and encoded from 200K oriented surface samples each.The data are curated from ObjaverseXL, ABO, 3DFUTURE, and HSSD.Source: C Implementation Details · p. 17
- N=128 slices per axis, window width w=8, and latent dimension C=512 yield 384 slice tokens per shape.The sparse decoder refines a 16^3 grid to 256^3 before differentiable Dual Marching Cubes extracts meshes.Source: C Implementation Details · p. 17
- The VAE trains for five epochs, while the rectified-flow transformer trains for 300K steps and uses 50 Euler steps at inference.Both models use eight NVIDIA A100 GPUs; the transformer is conditioned on frozen DINOv2 features.Source: C Implementation Details · p. 17
D Additional Results
Additional evaluations report strong open-surface reconstruction and plausible single-view generations on diverse in-the-wild objects, including shapes with thin or repeated structures.
Open Surface Evaluation
- CD falls from 0.05 to 0.04 versus SparseFlex, while F-Score@0.005 rises from 93.07 to 93.21 and F-Score@0.01 remains perfect.On DeepFashion3D, topology-aware metrics saturate near zero because the garments are topologically simple.Source: D Additional Results · p. 17
Image-to-3D in the Wild
- SILSA produces plausible 3D structure from single in-the-wild views and maintains consistency across rendered viewpoints.The results are strongest on geometry difficult for compact global latents, such as chairs, drones, motorcycles, and flowers.Source: D Additional Results · pp. 17–18, Figure 6 · p. 18
- Examples include chairs with legs and armrests, drones with thin supports, motorcycles with wheels and handles, and flowers with layered petals.These examples illustrate recovery of fine details alongside globally coherent assets.Source: D Additional Results · p. 18
E Ablations
The ablations examine slice resolution and overlap, topology supervision, axis count, and the Volumetric Anchor Lattice update rule, identifying choices that balance fidelity and structural consistency.
Slice representation
- N=128 provides the best efficiency-fidelity trade-off at 384 tokens; finer bins beyond N=128 saturate, while w=8 gives the strongest overall balance.N=1024 worsens Betti-Err, and w=16 slightly degrades performance, consistent with excessive slicing or aggregation weakening local evidence.Source: E Ablations · pp. 18–19, Table 6 · p. 19
Topology Loss
- The full topology objective gives the best balance between surface fidelity and topological correctness; LPH-only and Ltrans-only supervision each omit complementary constraints.LPH supervises topology within slices, whereas Ltrans aligns event locations along depth; the full objective combines both signals.Source: E Ablations · p. 19, Table 7 · p. 20
Number of Axes
- Betti-Err is 1.58 with three axes versus 2.61–2.74 for two-axis variants; single-axis z-only and x-only variants reach 5.92 and 6.18.The prose attributes the remaining two-axis gap to structures exposed by the third cross-sectional view.Source: E Ablations · p. 19, Table 8 · p. 20
VAL Update Mechanism
- Gated writes achieve the best results across all metrics; additive writes trail (CD 0.63, Betti-Err 1.74), while overwrite yields (CD 0.66, Betti-Err 1.91).Channel-wise gating learns how much existing lattice content to retain or replace as new slice evidence arrives.Source: E Ablations · p. 20
F Discussion
The discussion distinguishes topological correctness from geometric error, describes slice-level topology supervision, and notes limits tied to training data and input-image quality.
3D Topology Preservation
- 3D topology concerns connected components, holes, and their relationships, which geometric metrics can miss even when surface deviations are small.Betti numbers represent these properties: β0 counts components, β1 loops or tunnels, and β2 enclosed voids.Source: F Discussion · pp. 20–21
- SILSA matches persistence within cross-sections and aligns Betti transitions across neighboring slices, avoiding computationally prohibitive full-volume topology supervision.Persistent homology records feature birth, death, and persistence across a filtration; longer-lived features correspond to robust structures.Source: F Discussion · p. 21
Limitations
- Fidelity may decline for patterns far outside the training distribution or for ambiguous, low-information input views.The authors identify scaling to broader data sources as future work.Source: F Discussion · p. 21
Broader Impact
- The authors identify content creation, design, education, and simulation as potential applications, while noting risks of unauthorized replicas and displacement of manual modeling.They state that downstream applications should address these risks through appropriate safeguards.Source: F Discussion · p. 21