Source-linked AI summary
TTHRESH: Tensor Compression for Multidimensional Visual Data
Rafael Ballester-Ripoll, Peter Lindstrom, Renato Pajarola
TL;DR
Large multidimensional visualization data sets strain memory and network bandwidth, motivating lossy compression with controlled error. TTHRESH combines HOSVD with adaptive bit-plane coding of the core and factors, outperforming other compressors at visualization-relevant higher error ranges while degrading smoothly. Its compression efficiency and smooth degradation trade off against slower processing and costly random-access decompression.
Problem
Memory and network bandwidth constraints make efficient compression necessary for large multidimensional data sets used in visualization.
Method
TTHRESH uses Tucker/HOSVD decomposition with adaptive thresholding and bit-plane coding to compress full-rank core and factor coefficients.
Results
TTHRESH significantly outperforms other state-of-the-art compressors at higher error ranges used in visualization and achieves smooth degradation.
Takeaways & Limitations
The method supports fine compression control and linear manipulation such as filtering and subsampling in the compressed domain.
Takeaways & Limitations
TTHRESH is slower than speed-oriented compressors, and random-access decompression is relatively costly because the whole core must be traversed.
Abstract
from arXiv · showhide
Memory and network bandwidth are decisive bottlenecks when handling high-resolution multidimensional data sets in visualization applications, and they increasingly demand suitable data compression strategies. We introduce a novel lossy compression algorithm for multidimensional data over regular grids. It leverages the higher-order singular value decomposition (HOSVD), a generalization of the SVD to three dimensions and higher, together with bit-plane, run-length and arithmetic coding to compress the HOSVD transform coefficients. Our scheme degrades the data particularly smoothly and achieves lower mean squared error than other state-of-the-art algorithms at low-to-medium bit rates, as it is required in data archiving and management for visualization purposes. Further advantages of the proposed algorithm include very fine bit rate selection granularity and the ability to manipulate data at very small cost in the compression domain, for example to reconstruct filtered and/or subsampled versions of all (or selected parts) of the data set.
1 INTRODUCTION
TTHRESH addresses memory and transmission constraints in multidimensional visualization data by combining Tucker/HOSVD compression with adaptive, fine-grained coding. It reports strong performance at visualization-relevant error ranges and supports practical data manipulation and open-source use.
- Motivation: Multidimensional visualization applications need efficient lossy compression because memory and transmission bandwidth constrain large Cartesian-grid data sets.Relevant applications may require bounded error, fine rate granularity, random access, asymmetry, or support for arbitrary dimensionality.
- Approach: TTHRESH uses Tucker decomposition with adaptive thresholding and bit-plane coding to support arbitrary target accuracy.The approach also compresses factor matrices rather than only transform coefficients.
- Results: TTHRESH significantly outperforms other state-of-the-art compressors at higher error ranges commonly used in visualization, while remaining comparable at low error tolerances.The reported comparison concerns compression ratios and visualization-oriented operating ranges.
- Availability: The authors released an open-source C++ implementation usable as a standalone command-line utility or header-only library.The implementation is distributed under LGPL-3.0.
2 RELATED WORK
Prior scientific-volume compressors use curve fitting, prediction, truncation, fixed transforms, or data-dependent bases. Tucker-based methods improve transform-domain sparsity but must also store learned factor matrices.
- Scientific-volume compressors: ISABELA fits B-splines along a monotonic curve, while SZ predicts coefficients with low-degree polynomials or truncates IEEE 754 representations.These methods represent distinct linearization strategies for lossy scientific-volume compression.
- Fixed-basis transforms: Fourier, discrete cosine, and wavelet methods use designed transform bases to sparsify real-world signals.VAPOR integrates wavelet compression into interactive volume and flow exploration, while ZFP emphasizes fast random access and low error.
- Data-dependent transforms: PCA and Tucker decomposition use data-dependent factor matrices to improve transform-domain sparsity.The learned bases are comparatively small for three or more dimensions, but they must be stored.
3 TUCKER/HOSVD DECOMPOSITION
The Tucker model represents a tensor through factor matrices and a core tensor, while HOSVD constructs orthogonal factors from mode unfoldings. TTHRESH exploits the resulting coefficient structure but addresses limitations of coarse core truncation and fixed-width encoding.
- Tensor notation: Tensors are treated as multiarrays of dimension N ≥1, with mode unfoldings arranging fibers into matrices for tensor operations.The paper uses tensor-times-matrix products to express transformations along individual modes.
- The Tucker model: The full Tucker decomposition represents tensor T as a core tensor B transformed by factor matrices U(1), …, U(N) along each mode.The inverse transformation applies the inverses of the factor matrices to recover B.
- HOSVD: HOSVD constructs orthogonal Tucker factors from the left singular vectors of each mode unfolding and exists for any tensor shape and dimensionality.These factors act as uncentered PCA transformation matrices for mode fibers.
- Error and sparsity: For dimensions three and higher, the core contains far more coefficients than the factors and is therefore the main source of data reduction and error.Factor orthogonality preserves the tensor norm under the transform and helps analyze core-induced l2 error.
- Sparsifying properties: HOSVD coefficients tend to be quasi-sparse, and the transform can preserve the core under certain spatial permutations or movements of the data.The decomposition decorrelates data across spatial scales without explicit partitioning or predefined multiresolution filter banks.
- Core truncation and its limitations: Traditional truncation discards trailing factor columns and corresponding core slices, but this slice-wise strategy has coarse granularity compared with coefficient-wise elimination.Orthogonality also makes the MSE from zeroing a coefficient proportional to its squared magnitude.
4 PROPOSED ALGORITHM
TTHRESH compresses multidimensional data by applying a full HOSVD, encoding its core and factor matrices, and stopping core bit-plane transmission at a user-specified error target. Its coding combines lossless RLE and arithmetic coding for leading bits with verbatim storage for trailing bits, while factor compression addresses a potential overhead of Tucker decompositions.
- 4.1 Error specification: TTHRESH accepts a user-defined error target specified through relative error, root-mean-square error, or peak signal-to-noise ratio.The selected target is converted internally to a sum of squared errors (SSE).
- 4.1 HOSVD transform: The pipeline computes a full HOSVD, flattens the core into C coefficients, converts them to 64-bit integers, and represents them as a C × 64 binary matrix.The HOSVD produces orthogonal factor matrices and a core with the original tensor size before coefficient coding.
- 4.2 Bit-plane coding: Bit-plane coding transmits columns from the most significant plane downward and stops when the reconstructed core reaches the prescribed SSE tolerance.The strategy gives equal importance to bits within a plane, and an alternative file-size stopping criterion could also be used.
- 4.2 Bit-plane coding: Leading zero runs are compressed losslessly with RLE followed by arithmetic coding, whereas trailing bits are stored verbatim because they are nearly uniformly random.A significance mask records coefficients whose leftmost 1 bit has already been encountered, determining which coding mode applies.
- 4.3 Factor compression: Factor matrices can create significant overhead despite being small relative to the core, motivating careful factor compression alongside core compression.For a 256^3 volume, the factors comprise about 1% of the full HOSVD element count, but their overhead can become important if left insufficiently compressed.
5 DECOMPRESSION AND POST-PROCESSING
TTHRESH reconstructs data by reversing the HOSVD transform, while compressed-domain filtering and decimation operate directly on factor matrices before decompression.
- Decompression: Decompression reverses the transmitted HOSVD core bit planes and factors, then reconstructs the tensor through N tensor-times-matrix products.Zero core slices are discarded to accelerate reconstruction.
- Compression-domain Resampling: Filtering compressed tensors uses convolution on factor matrices, exploiting multilinearity without changing the Tucker core.Separable filters are applied as column-wise convolutions with the corresponding factor matrices.
- Compression-domain Resampling: Compressed-domain decimation supports downsampling, box filtering, and separable Lanczos-2 filtering before row subselection.Lanczos-2 convolves factor columns with a 1D three-lobed kernel using five samples and window parameter 2.
- Compression-domain Resampling: Index ranges and strides enable previews, subvolume selection, slicing, dimension reversal, and frame-by-frame visualization while leaving the Tucker core unchanged.Filtering and downsampling require O(NI^2 log I) operations for column-wise factor convolution.
6 RESULTS
Experiments across 14 volume data sets compare TTHRESH with four state-of-the-art compressors, evaluating quality, visual degradation, spectral behavior, speed, and compressed-domain decimation.
- Evaluation Setup: 14 data sets were evaluated against TRUNC, ZFP, SZ, and SQ using PSNR-versus-compression-ratio curves and visual renderings.The test collection included integer and floating-point volumes plus two time-varying volumes.
- Quality Results: TTHRESH performs similarly or sometimes worse at lower compression ratios, then surpasses competing methods after a data-set-dependent tipping point.Its advantage widens as compression ratios increase.
- Visual Results: At medium to high compression ratios, TTHRESH preserves coarse features while smoothing or removing smaller details.Block-like artifacts appear only at exceedingly high ratios, later than with other algorithms.
- Spectral Results: TTHRESH acts as a low-pass filter because insignificant core coefficients commonly correspond to high-frequency trailing factor columns.Fourier observations align with the smooth low-frequency artifacts seen under heavier compression.
- Spectral Results: SZ, SQ, and ZFP shift Fourier spectra toward higher frequencies, opposite to the low-frequency shift observed for TTHRESH.The comparison is shown through Fourier-magnitude histograms at different compression rates.
- Computational Speed: TTHRESH is 0.5–2 orders of magnitude slower than ZFP but generally faster than SQ for the measured compression tasks.Average compression/decompression times were 2.4s/1.0s for Teapot and 61.5s/25.8s for Isotropic-fine.
- Compressed-domain Decimation: Lanczos-2 outperforms the other implemented decimation methods in the compressed-domain visual experiment.The experiment compares visual results after factor-matrix decimation followed by decompression.
7 DISCUSSION
TTHRESH combines competitive low-to-medium-ratio accuracy with stronger medium-to-high-ratio performance and smooth, finely controllable degradation for visualization.
- Quality and Visualization: TTHRESH consistently outperforms other compressors at medium to high compression ratios while retaining competitive accuracy at lower ratios.The overtaking point typically already produces renderings close to visually indistinguishable from the original, especially for higher bit depths.
- Degradation and Control: Global bases produce smooth degradation through a Fourier-spectrum shift and gradual erosion of smaller details and features.Transmission may stop at any point within any bit plane, providing fine-grained control over final error.
- Post-processing: Compressed-domain filtering and decimation extend TTHRESH beyond storage toward efficient visualization post-processing.The supplied discussion states this capability after describing fine-grained bit-plane stopping and error bounds.
Limitations
TTHRESH prioritizes compression rates and smooth degradation, but its monolithic transform core makes it slower and increases random-access decompression cost. Splitting data into bricks is suggested as a possible speed improvement.
- TTHRESH is slower than speed-oriented compressors such as ZFP because its transform core is monolithic.The authors identify compression and decompression speed, especially compression, as a secondary priority.
- Random-access decompression is relatively costly because the whole core must be traversed in every case.
- Compression quality curves compare TTHRESH with TRUNC, ZFP, SZ, and SQ across 12 example volumes and varying compression ratios.The plotted ratios reach up to 80:1 for integer data and 200:1 for floating-point data; gray regions mark ratios where TTHRESH has the highest PSNR.
- Compression quality curves are also shown for two time-varying volumes.The figure is presented as a companion to the 12-volume comparison.
- Brick-by-brick data splitting is proposed as a way to improve compression and decompression speed.The proposal follows the spirit of tensor-compressed multiresolution rendering systems.
8 CONCLUSION
TTHRESH is a tensor-decomposition compressor designed for high-ratio data reduction in storage and visualization. It retains all ranks, compresses bit planes up to a threshold, and supports compressed-domain linear manipulation, while future work targets speed and progressive decoding.
- TTHRESH is a tensor-decomposition compression algorithm emphasizing data reduction at high compression ratios for storage and visualization.
- Unlike previous HOSVD-driven approaches, TTHRESH keeps all ranks and applies lossless compression to bit planes up to a threshold.
- Factor orthogonality makes all coefficients contribute equally to final l2 error, allowing the full core to be sorted as one block.
- The algorithm supports linear manipulation of the data set in the compressed domain.
- The authors prioritize data-reduction rates over general compression and decompression speed.They identify eigensolver iteration limits and discarding unimportant core slices as possible speed improvements with an accuracy cost.
- Progressive decompression is compatible with the coder, but coefficient signs and factor columns would need to be encoded progressively.The authors leave these implementation possibilities for future investigation.
- At equivalent PSNR, four example volumes generally require half or less the space of the other four tested methods.The exception is the Foot scan at higher quality, where performance is similar.