Source-linked AI summary
Thread-Efficient Decoding for Neural Texture Compression
Janarbek Matai, Sho Ikeda, Lukasz Lipski, Takahiro Harada
TL;DR
Neural texture compression saves texture memory but suffers runtime losses from GPU thread divergence. The paper uses shared MLP decoders, gradual decoder freezing, and texture clustering, finding reduced divergence with a quality–efficiency trade-off across large texture and scene evaluations.
Problem
Neural texture compression reduces texture memory but incurs runtime costs because rendering must query different MLP weights, causing GPU thread divergence.
Method
The paper shares decoders across texture groups, trains them with decoder freezing, and clusters semantically similar textures to balance divergence reduction and reconstruction quality.
Results
Across 586 textures and three scenes, similarity-based grouping performs best, while shared-decoder configurations trade lower divergence against reconstruction quality.
Takeaways & Limitations
Shared decoding reduces thread divergence in standalone compression and rendering scenarios while retaining effective reconstruction when trained appropriately.
Takeaways & Limitations
The study uses OpenAI CLIP for semantic grouping, while alternative low-level-feature clustering methods and systematic sensitivity analysis of the cluster count remain future work.
Abstract
from arXiv · showhide
Neural texture compression (NTC) achieves higher compression ratios than BCn formats but suffers from GPU thread divergence, which significantly reduces runtime performance. In this work, we propose a shared decoder MLP architecture -- trained with a gradual decoder freezing schedule -- combined with texture clustering to reduce thread divergence by 25%-52% while preserving rendering quality. We evaluate our method on over 500 textures and multiple real rendering scenes, demonstrating up to 8.48x speedup on the Radeon RX 9070 XT GPU compared to non-shared baselines. Our key contributions include: (1) a unified shared decoder architecture that reduces divergence by grouping textures; (2) a training recipe with gradual decoder freezing that improves stability and reconstruction accuracy; (3) a semantic clustering strategy using CLIP embeddings that groups similar textures for effective decoder sharing; and (4) comprehensive performance and ablation studies validating our approach.
1 Introduction
Neural texture compression reduces texture memory but introduces runtime costs from GPU thread divergence. This paper addresses that trade-off with shared decoders, training strategies, and texture clustering.
- Motivation: Neural texture compression achieves higher compression ratios than traditional analytical methods but increases texture memory efficiency challenges at runtime.Its runtime implementation must query different MLP weights during rendering.
- Motivation: GPU efficiency drops when multiple decoder networks are evaluated within the same wave, causing thread divergence during neural texture rendering.Tensor operations execute across an entire wave rather than independently per lane.
- Approach: A shared decoder serves multiple texture sets, shifting adaptation to individual encoders while reducing divergence; a single decoder eliminates divergence but reduces rendering quality.The paper presents shared decoding as a trade-off between runtime efficiency and reconstruction quality.
- Evaluation: The study evaluates shared decoding on more than 500 textures, real scenes, clustering strategies, and training or network-size ablations.Machine-learning-based clustering is reported as outperforming baseline and random grouping.
- Results: Rendered scenes using the clustered shared-decoder method have nearly indistinguishable visual quality from default textures while exhibiting less thread divergence.The paper reports almost 5X−8X faster texture decompression in this context.
- Positioning: The method is presented as orthogonal to existing NTC approaches and as a trade-off between reconstruction accuracy and reduced thread divergence.Prior work primarily focused on cooperative vectors rather than shared decoders.
2 Thread Divergence in NTC
The paper defines thread divergence through texture variation within GPU blocks and introduces maps and scores to quantify it. Shared decoding and clustering are proposed to reduce divergence while avoiding the quality loss of one universal decoder.
- Definition: Thread divergence occurs when threads in one GPU wave take different execution paths, including when pixels use different texture-specific MLP decoders.The paper treats divergence as a runtime bottleneck in neural texture compression.
- Divergence map: The divergence map records the number of distinct texture IDs accessed by each scene block.Each block corresponds to a region of the rendered scene.
- Divergence map: A block value of 1 indicates low divergence because all pixels use one texture, whereas larger values indicate higher divergence.The map therefore exposes spatial variation in texture access.
- Quantification: The divergence score measures average absolute deviation from an all-ones map and is zero exactly when every block uses one texture.The metric is invariant to matrix size and increases with entry magnitude and variability.
3 Methodology
The methodology combines clustered shared decoders with gradual decoder freezing to reduce divergence while retaining reconstruction quality. Textures are embedded with CLIP, grouped by k-means, and trained with group-specific shared decoders and texture-specific encoders.
- Clustering and decoder sharing: Clustering reduces the number of decoders while preserving better quality than a single decoder by grouping semantically related textures.The method treats single, random, and clustered sharing as alternatives that trade reconstruction accuracy against thread divergence.
- Training shared decoders: Gradual decoder freezing smoothly reduces the decoder’s effective learning contribution after a chosen training epoch.The schedule avoids abrupt freezing and is intended to improve training stability and final accuracy.
- Texture clustering: The clustering pipeline extracts 512-dimensional CLIP features, applies k-means with m clusters, and assigns each texture to one group.The resulting assignments define texture subsets T_g and group sizes k_g for group-wise training.
- Group-wise encoder–decoder networks: Each cluster trains texture-specific encoders with one decoder shared across all textures in that cluster.There are m decoders in total, and training minimizes reconstruction loss over the textures in each group.
4 Results
The evaluation covers 586 textures and three real rendering scenes, comparing baseline, shared, clustered, random, and category decoder architectures. Semantic clustering and decoder freezing improve the quality–divergence trade-off, while shared decoding delivers substantial GPU speedups.
- Experimental setups: 586 textures were evaluated on Polyhaven, alongside Yokohama, Bistro, and ToyShop rendering scenes.The scene experiments used 107, 104, and 117 textures respectively, with 10 clusters per scene.
- Texture accuracy: Clustered sharing achieved lower PSNR than Baseline but higher PSNR than Random, Category, and Single across the 586-texture evaluation.Both Clustered and Random used the same number of textures per group, supporting a comparison of grouping strategy.
- Rendering quality and divergence: Shared-decoder architectures substantially reduced thread divergence while maintaining acceptable accuracy in real rendering scenes.The results describe this as a favorable trade-off between divergence reduction and reconstruction quality.
- Rendering quality and divergence: 25%, 25%, and 52% fewer divergent threads were measured for Yokohama, Bistro, and ToyShop, respectively, using Clustered versus Baseline.The corresponding Clustered counts were 11,222, 13,373, and 6,407 divergent threads.
- Ablation study: Freezing the shared decoder improved training stability and accuracy, with freezing around 50% of total epochs usually producing the best accuracy.Accuracy initially drops when freezing occurs, then surpasses training without freezing after several epochs.
- Performance analysis: 848% and 795% average speedups were obtained over Non-Shared WMMA and Non-Shared FMA, respectively, for 10 decoders at 1.6 GHz.Measurements used FP16 HLSL implementations on a Radeon RX 9070 XT across multiple output resolutions.
5 Discussion and Conclusion
The paper combines shared decoder architecture with clustering to reduce thread divergence in neural texture compression, evaluating accuracy, theoretical divergence reduction, performance, and ablations. Results identify similarity-based grouping as the strongest evaluated balance, while cluster count remains an unresolved trade-off between divergence reduction and reconstruction quality.
- The proposed shared decoder architecture and clustering-based method target thread divergence in neural texture compression across public datasets and rendered scenes.The evaluation includes accuracy, theoretical divergence reduction, performance, and ablation studies.
- Shared decoders can perform well when trained with appropriate layer freezing, reducing thread divergence in both standalone compression and rendering scenarios.The results identify training strategy as important to shared-decoder effectiveness.
- Similarity-based grouping yields the best performance among 586 textures and three scenes, while random grouping is slightly worse on average.A single decoder minimizes thread divergence but produces the lowest PSNR, revealing an accuracy–divergence trade-off across grouping strategies.
- The study uses OpenAI CLIP to group textures by semantic similarity, while alternative low-level-feature clustering methods remain future work.Suggested alternatives include texture frequency and edge content.
- The number of clusters m governs the trade-off between divergence reduction and reconstruction quality, but systematic sensitivity analysis remains future work.