Source-linked AI summary
Implicit Neural Representation for Hyperspectral Video Compression
Alfredo Scalera, Paul Murray, Jaime Zabalza
TL;DR
HS video compression remains in its early stages, with image methods often applied frame by frame. This paper adapts an INR-based video-compression model for HS video, reporting improved rate-distortion results and low-bitrate tracking performance over PCA+JPEG2000.
Problem
HS video compression remains early-stage, and image-compression methods for HS data have mainly been applied frame by frame.
Source: §1 · p. 1Method
The paper adapts HNeRV, a hybrid RGB approach that treats video compression as model and latent compression, for HS videos [5].
Source: §3.2 · p. 2Results
Against PCA+JPEG2000, HS-HNeRV achieves a BD-PSNR of 4.99 dB and BD-rate of −88.88%, and at approximately 0.13 bpp improves tracking AUC by 23.42% and DP by 35.56%.
Source: §4 · p. 3Takeaways & Limitations
The results demonstrate potential for INR-based overfit compression in low-data HS video settings, without requiring large amounts of data for a general model.
Source: §5 · p. 4Performance depends strongly on hyperparameter choice, and searching across multiple videos can defeat dataset-free compression when input distributions differ.
Source: §5 · p. 4
Abstract
from arXiv · showhide
With the advent of snapshot cameras, hyperspectral video is becoming more readily available. In recent years, new applications have emerged which have led to increasingly larger datasets. However, hyperspectral video compression remains in the early stages. In this study, we explore the use of implicit neural representation as a candidate solution. We propose a novel extension of an existing RGB video compression model, achieving Bj{\o}ntegaard Delta PSNR gains of +4.99 dB and Bj{\o}ntegaard Delta rate of -88.88% compared to traditional hyperspectral image compression methods applied frame-by-frame. In addition to reconstruction quality, the effects on downstream task performance are measured in the form of object tracking success. Compared to video compressed with methods based on principal component analysis and JPEG2000 in low data regimes, our proposed method improves tracking area under the curve by up to 23.42% and distance precision by up to 35.56% on examples from the HOT2026 dataset.
1. INTRODUCTION
Hyperspectral video is becoming more prevalent, but compression remains at an early stage and has mainly used image-compression methods frame by frame. The paper proposes implicit neural representation (INR) for HS video compression and compares it with PCA-based approaches.
1. INTRODUCTION
- HS video compression remains in its infancy, with HS image-compression methods mainly applied frame by frame.Snapshot sensors have also brought varied applications, including object tracking and scene segmentation.Source: §1 · p. 1
- The paper proposes INR for HS video compression, extending an INR-based compression method and comparing it with common PCA-based approaches for HS imagery.It also aims to quantify gains over current practices.Source: §1 · p. 1
- INR-based methods use neural networks' data-representation capabilities while avoiding the need for large datasets [5].Source: §1 · p. 1
2. RELATED WORK
Related work spans established RGB codecs, learned video compression, and HS video coding that has largely built on image methods. The paper positions overfit neural compression as an unexplored approach for HS video.
2. RELATED WORK
- RGB video compression is a mature field, with systems commonly built on classical hybrid codecs such as AVC, HEVC, and VVC.Source: §2 · p. 1
- Learned video codecs include general models trained end-to-end on large datasets, and overfit codecs such as HNeRV [5], NVRC, and COOL-CHIC.Source: §2 · p. 1
- Snapshot-camera advances have accompanied the release of HS video datasets for object tracking and automated driving, including HOT2020, HOT2026, IMEC25, and HSI-Drive.Source: §2 · p. 1
- HS video compression commonly processes each datacube frame individually using spectral correlation, [16], while prior work also added temporal compensation to an HS image coder.Sparse Tucker tensor decomposition has also been applied per frame.Source: §2 · pp. 1–2
- Overfit neural compression methods had, to the authors' knowledge, not yet been applied to HS video compression.The work explores this approach, specifically targeting INR.Source: §2 · p. 2
3. METHODS
The methods adapt HNeRV for hyperspectral video and compare it with per-frame PCA+JPEG2000 compression. Evaluation covers reconstruction fidelity and object tracking on HOT2026 video sequences.
3.1. PCA-based Video Compression
- PCA is fitted per frame to reduce each datacube to three channels, which JPEG2000 compresses at a target bits-per-pixel rate.The PCA projection matrix and mean vector are min-max quantized, with minima and scale factors stored as bitstream side information.Source: §3.1 · p. 2
3.2. INR-based Video Compression
- HS-HNeRV adapts HNeRV with multi-channel encoder and decoder networks, HSFusion loss, and hyperparameter search for HOT2026.The decoder is shared across frames, while each frame has unique latent variables that Huffman coding losslessly compresses after quantization.Source: §3.2 · p. 2, Figure 2 · p. 3
- HSFusion combines L2 and cosine-similarity losses to target general reconstruction and per-pixel spectral fidelity, respectively.The loss weights these components with λ ∈ [0, 1].Source: §3.2 · p. 2
3.3. Experiments
- The evaluation measures decompressed-frame fidelity with PSNR and SAM, and downstream tracking with AUC and distance precision.Tracking uses CSSTrack; AUC measures success-plot area, while distance precision counts frames with center error below 20 pixels.Source: §3.3 · pp. 2–3
- Four HOT2026 test sequences with 16 spectral bands were center-cropped from 512 × 272 to 512 × 256 pixels.The crop provides a 2:1 aspect ratio for the model.Source: §3.3 · p. 3
- Fidelity metrics were averaged across frames after sweeping model sizes and PCA+JPEG2000 target rates; tracking comparisons matched bitrate.Tracking used three HS-HNeRV model sizes and a CSSTrack checkpoint trained on HOT20202.Source: §3.3 · p. 3
4. RESULTS
HS-HNeRV improves reconstruction quality over PCA+JPEG2000, with strong gains in rate-distortion performance and low-bitrate tracking. Tracking results favor PCA+JPEG2000 at lower compression rates.
4. RESULTS
- −88.88% BD-rate and +4.99 dB BD-PSNR versus PCA+JPEG2000 quantify HS-HNeRV’s rate-distortion gains across four test sequences.The comparison averages results across the four test sequences; Fig. 3 reports higher PSNR and lower SAM over equivalent bpp ranges.Source: §4 · p. 3, Figure 3 · p. 3
- At approximately 0.13 bpp, HS-HNeRV improves tracking AUC by 23.42% and DP by 35.56% over PCA+JPEG2000.Table 1 reports CSSTrack performance on compressed HOT2026.Source: §4 · p. 3, Table 1 · p. 4
- At lower compression rates, PCA+JPEG2000 surpasses HS-HNeRV; CSSTrack DP on PCA+JPEG2000 at 0.6 bpp also exceeds its raw-data result.Source: §4 · p. 3
5. DISCUSSION
HS-HNeRV reaches the same PSNR as PCA+JPEG2000 with less data, while HSFusion supports pixel-level spectral consistency. The approach remains sensitive to hyperparameter choices when input distributions differ.
5. DISCUSSION
- HS-HNeRV achieves the same PSNR as PCA+JPEG2000 while requiring only 11.12% of the data.The discussion reports stronger representation across all bitrates.Source: §5 · p. 4
- HSFusion combines L2 and cosine-similarity losses to encourage spectral consistency for pixel-level algorithms.Fig. 4 compares spectral fidelity at equivalent PSNR across hyperparameter-search trials.Source: §5 · p. 4, Figure 4 · p. 4
- Hyperparameter search across multiple videos can undermine dataset-free compression when input distributions differ.The discussion suggests Bayesian or gradient-based optimization as possible mitigation.Source: §5 · p. 4
6. CONCLUSION
The conclusion presents INR as a strong approach to HS video compression, improving reconstruction quality over PCA combined with JPEG2000. Future work includes HS-specific architectures, hyperparameter optimization, and updated quantization and encoding procedures.
6. CONCLUSION
- INR-based compression improves reconstruction quality by +4.99 dB compared with PCA combined with JPEG2000.Source: §6 · p. 4
- Future work will explore bespoke HS video architectures, hyperparameter optimization for overfitting, and quantization and encoding aligned with newer overfitting RGB codecs.Source: §6 · p. 4