PaperlayerView exact revision on arXiv

Source-linked AI summary

Implicit Neural Representations for Image Compression

Yannick Strümpler, Janis Postels, Ren Yang, Luc van Gool, Federico Tombari

arXiv:2112.04267v2eess.IVcs.CVcs.LG

TL;DR

The paper addresses whether Implicit Neural Representations can serve as effective image-compression tools despite their slow encoding. It develops a complete INR compression pipeline with meta-learned initialization, and reports strong performance against prior INR methods and traditional codecs, while remaining weaker on high-resolution images than RDAEs and BPG.

  • Problem

    Prior INR research mainly optimized reconstruction, leaving their use as a comprehensive image-compression method comparatively underexplored despite the importance of compact image storage.

  • Method

    The paper builds an INR-based compression pipeline combining input encoding, quantization, quantization-aware retraining, entropy coding, and MAML-based meta-learned initialization.

  • Results

    The approach vastly outperforms COIN, is competitive with traditional image codecs, and meta-learned initialization reduces training time by up to 90% at fixed performance.

  • Takeaways & Limitations

    INR compression is competitive with traditional codecs over a large portion of bitrates and can extend to coordinate-based modalities such as 3D data.

  • Takeaways & Limitations

    Performance is competitive on lower-resolution images but falls short of RDAEs and BPG on higher-resolution images, likely because independent pixel processing scales inefficiently.

Abstract

from arXiv · show

Recently Implicit Neural Representations (INRs) gained attention as a novel and effective representation for various data types. Thus far, prior work mostly focused on optimizing their reconstruction performance. This work investigates INRs from a novel perspective, i.e., as a tool for image compression. To this end, we propose the first comprehensive compression pipeline based on INRs including quantization, quantization-aware retraining and entropy coding. Encoding with INRs, i.e. overfitting to a data sample, is typically orders of magnitude slower. To mitigate this drawback, we leverage meta-learned initializations based on MAML to reach the encoding in fewer gradient updates which also generally improves rate-distortion performance of INRs. We find that our approach to source compression with INRs vastly outperforms similar prior work, is competitive with common compression algorithms designed specifically for images and closes the gap to state-of-the-art learned approaches based on Rate-Distortion Autoencoders. Moreover, we provide an extensive ablation study on the importance of individual components of our method which we hope facilitates future research on this novel approach to image compression.

1 Introduction

This work investigates Implicit Neural Representations (INRs) as a novel paradigm for image source compression. It addresses INR compression’s competitiveness and encoding-time challenges with meta-learned initializations and a comprehensive compression pipeline.

  • Source compression trades file size against preserved quality, making rate-distortion performance central for lossy image compression.
  • INRs represent grid-based data by learning coordinate-to-value mappings and are proposed as flexible, potentially compact representations for images.
  • INR compression faces two challenges: straightforward methods struggle against traditional algorithms, and overfitting makes encoding impractically slow.
  • The method uses MAML-based meta-learned initializations to reach target representations with fewer gradient updates and improve rate-distortion performance.
  • The proposed pipeline vastly outperforms COIN, competes with traditional image codecs, and meta-learned INRs outperform JPEG2000 and some RDAEs on downsampled images.

2 Related Work

Prior learned image compression predominantly uses Rate-Distortion Autoencoders, while INR research spans representations for images, scenes, and 3D shapes. COIN introduced INR-based image compression but omitted several components developed here.

  • Learned Image Compression: Rate-Distortion Autoencoders jointly optimize decoded sample quality and encoded file size using constrained autoencoder architectures.
  • Learned Image Compression: Later RDAE work improves compression with hyperpriors, autoregressive entropy models, attention, Gaussian mixture models, and variable-rate designs.
  • Implicit Neural Representations: INRs have been applied to represent 3D shapes, scenes, images, and other compact data through coordinate-based neural fields.
  • INR-based Compression: COIN overfits INR weights for individual images and compresses them using quantization, but does not use meta-learning, SIREN positional encodings, post-quantization retraining, or entropy coding.

3 Method

The method represents images with coordinate-based neural networks and compresses their weights through overfitting, quantization, retraining, and entropy coding. Meta-learned initializations and input encoding target rate-distortion quality and encoding efficiency.

  • 3.1 Background: INRs approximate images as continuous coordinate-to-RGB functions, allowing evaluation on arbitrary normalized coordinates and reconstruction on uniform grids.
  • 3.1 Background: Unlike RDAEs, INRs store image information implicitly in network weights; encoding trains the network, while decoding loads weights and evaluates coordinates.
  • 3.2 Image Compression using INRs: SIREN uses sine activations with frequency ω = 30, while model size is varied to obtain rate-distortion curves.
  • 3.2 Image Compression using INRs: Input encoding expands coordinates with sine and cosine frequency terms, concatenated with the original coordinate; L controls the number of frequencies.
  • 3.3 Compression Pipeline for INRs: The pipeline overfits an INR using MSE, quantizes its weights, applies AdaRound and quantization-aware training, then entropy-codes the quantized weights.
  • 3.3 Compression Pipeline for INRs: MAML learns an image-distribution-informed initialization so encoding starts near useful weights and transmits the update Δθ relative to θ0.
  • 3.3 Compression Pipeline for INRs: Weight updates can have smaller value ranges than full weights, reducing quantization steps and average rounding error at fixed bitwidth.
  • 3.4 INRs for 3D Shape Compression: The same INR compression pipeline applies to coordinate-based 3D shape data with minimal modification.

4 Experiments

Experiments evaluate INR compression on Kodak, CelebA, and 3D shapes using bitrate, PSNR, and task-specific comparisons. Meta-learning improves convergence and rate-distortion performance, while positional encoding and sine activations provide strong architectural choices.

  • Datasets and metrics: Experiments evaluate Kodak, CelebA, and reduced-resolution Kodak images, alongside a 3D shape compression transfer experiment.Kodak contains 24 images, CelebA evaluation uses 100 test images, and the 3D experiment uses five high-resolution meshes.
  • Datasets and metrics: Bitrate is measured as total stored bits per pixel, while distortion is measured with MSE and reported as PSNR in decibels.The experiments compare traditional codecs, COIN, and RDAE-based image compression methods.
  • Full-resolution comparison: The basic INR approach outperforms COIN across bitrates, while the meta-learned approach improves over the basic approach at every bitrate on full-resolution datasets.Meta-learned INRs reach JPEG2000 performance at the lowest examined bitrate but are surpassed by JPEG2000 at higher bitrates; BPG and the strongest RDAE outperform them on both datasets.
  • Reduced image resolution: On reduced-resolution Kodak images, meta-learned INRs outperform the basic approach and all compared methods except BPG and Xie et al. across the bitrate range.The comparison uses images downsampled by factors of two and four.
  • Visual comparison: Visual comparisons show better local detail and tonal transitions than JPEG2000 in some cases, but the meta-learned method introduces periodic sky artifacts and edge smearing.JPEG exhibits heavy blocking and color loss, while JPEG2000 shows blurring or blocking around edges and high-frequency details.
  • Convergence speed: Meta-learning reduces training time by 90%, matching the basic approach’s 25,000-epoch performance after 2,500 epochs.The convergence advantage is especially pronounced early in overfitting, with better performance after three epochs than the basic approach achieves after 50 epochs.
  • Input encoding and activation: Sine activations outperform ReLU across configurations, positional encoding beats Gaussian encoding, and positional encoding makes SIREN more robust to quantization.The architecture study evaluates multiple hidden dimensions on Kodak; both sine and ReLU show reduced quantization error with positional encoding, most clearly for SIREN.

5 Conclusion

INRs show strong potential for compression, with meta-learned initializations, SIREN, and positional encodings improving rate-distortion performance. The method is competitive with traditional codecs across many bitrates, transfers to 3D shapes, but remains weaker on high-resolution images.

  • The approach is the first INR-based method competitive with traditional codecs over a large portion of bitrates.
  • Meta-learned initializations reduce bitrate at the same reconstruction quality, with larger gains when training and test image distributions are more similar.The effect is especially strong on CelebA, whose face distribution has less variation than natural scenes.
  • Meta-learned initializations reduce training time by up to 90% at fixed performance.
  • Input encodings improve INR compression and make representations more robust to quantization.The result highlights the importance of inductive-bias choices for compression.
  • Performance is competitive on lower-resolution images but falls short of RDAEs and BPG on higher-resolution images.The authors hypothesize that independent pixel processing scales inefficiently and motivate INR architectures beyond MLPs.
  • The basic approach outperforms Draco on 3D mesh compression, demonstrating transferability beyond images.

7 Supplementary Material

The supplementary material describes meta-learning for INR image regression and the subsequent overfitting procedure. The algorithm learns both model initializations and adaptable inner-loop learning rates through nested optimization.

  • The supplementary material covers overfitting from meta-learned initializations, architecture choices, L1 regularization, quantization bitwidth, generalization, and runtime comparisons.
  • 7.1 Meta-Learning Algorithm: Meta-learning samples images and coordinate grids, adapts INR parameters through inner-loop updates, and optimizes the initialization across outer-loop iterations.The initialization is trained for image regression rather than signed distance function regression.
  • 7.1 Meta-Learning Algorithm: The inner loop minimizes image reconstruction MSE and updates parameters using learning rates represented by α.
  • 7.1 Meta-Learning Algorithm: Meta-learning can learn a separate learning rate for every parameter and inner-loop step, yielding k times as many learning rates as model parameters.
  • 7.1 Meta-Learning Algorithm: After k inner-loop iterations, the outer loop backpropagates through the adaptation process to update the meta-learned weights and learning rates.

7.2 Overfitting from Meta-Learned Initializations

Overfitting begins from parameter-wise meta-learned learning rates and reaches a near-final image in three inner-loop steps before continuing with Adam. A 100-epoch warmup prevents early degradation and improves higher-bitrate performance.

  • At the start of overfitting, parameter-wise learning rates from meta-learning bring the INR close to the final image in k = 3 steps.Optimization then continues with Adam, whose momentum terms are initially uninitialized.
  • A linearly increasing learning rate during a warmup phase of 100 epochs prevents initial reconstruction-quality degradation.
  • Warmup improves performance at higher bitrates compared with training without warmup.

7.3 Number of Layers and Hidden Dimension

The supplementary experiments vary MLP depth, hidden dimension, and L1 regularization while examining quantization bitwidths. They show that regularization can reduce entropy but must be balanced against excessive weight restriction.

  • 7.3 Number of Layers and Hidden Dimension: MLP depth and width both change parameter count and therefore indirectly affect bitrate.The study evaluates hidden units M ∈ {32, 48, 64, 96, 128} and hidden layers from 2 to 8.
  • 7.3 Number of Layers and Hidden Dimension: Compression performance is compared across models with 2–8 hidden layers and quantization bitwidths of b = 7 or b = 8 bits.
  • L1 Regularization: λ = 10^-5 performs better at higher bitrates than lower L1-regularization choices, appearing as a bitrate reduction.
  • L1 Regularization: λ = 10^-4 restricts weights too much and performs worse than λ = 10^-5.
  • L1 Regularization: L1 regularization can reduce entropy but should be combined with an architecture-size modification for a good rate-distortion trade-off.

7.5 Post-Quantization Optimization.

Post-quantization optimization improves INR compression, with the strongest performance obtained by combining AdaRound and retraining. Bitwidth also affects rate-distortion performance differently for meta-learned and basic approaches.

  • AdaRound and retraining each consistently improve performance, while their combination performs best across the bitrate range.
  • 7-bit quantization is best for the meta-learned approach on both datasets.
  • 8-bit quantization outperforms lower bitwidths for the basic approach, although Kodak shows only a small difference between 7 and 8 bits.
  • At 8 bits, most tested configurations nearly recover unquantized performance.
  • Meta-learned initializations still outperform basic initializations on cropped and resized Kodak images outside the CelebA training distribution.

7.8 Further Runtime Comparison

The runtime study evaluates whether INR compression can trade rate-distortion performance for faster encoding and decoding under a JPEG-matching constraint.

  • Runtime is compared with JPEG and Xie et al. while requiring rate-distortion performance to match JPEG.
  • The method is normally trained to full convergence for best rate-distortion performance, but can be tuned for faster runtime.

Image Compression Hyperparameters

The image-compression setup varies INR architecture size and training procedures across resolutions, meta-learning, overfitting, quantization, and entropy coding.

  • Architecture: Full-resolution Kodak uses M ∈ {32, 48, 64, 128}, while full-resolution CelebA uses M ∈ {24, 32, 48, 64}.
  • Architecture: Reduced-resolution Kodak uses smaller hidden dimensions and reduces input frequencies to L = 12 at half resolution.
  • Meta-Learning the Initializations: Meta-learning validates on 100 images from CelebA or DIV2K and saves the initialization with the lowest validation loss.
  • Overfitting: Overfitting uses the training image itself for validation, treats one epoch as one optimizer step, and trains for 25000 epochs.
  • Quantization, Post-Quantization Optimization & Entropy Coding: The default quantization pipeline combines AdaRound and quantization-aware training before entropy coding.

3D Shape Compression Hyperparameters

The 3D shape pipeline adapts INR overfitting from image pixels to sampled surface-distance data and applies quantization and post-quantization optimization. Visual comparisons report smoother and more faithful reconstructions than Draco at similar or lower storage.

  • 3D Shape Compression Hyperparameters: 3D compression fits the INR using 100000 point samples with ground-truth distances to the surface, training on batches of 10000 points.
  • 3D Shape Compression Hyperparameters: The 3D setup uses M ∈ {32, 48, 64, 128}, L = 16 input frequencies, 8-bit quantization, AdaRound, and retraining.
  • 7.11 Visualization of compressed 3D shapes: INR reconstructions produce smoother surfaces and preserve shape detail more faithfully than Draco, whose compression introduces significant surface noise.
  • 7.11 Visualization of compressed 3D shapes: The M = 64 model requires roughly one quarter of the M = 128 model’s storage while retaining a similar overall appearance with smoothed details.
  • 7.11 Visualization of compressed 3D shapes: Figure 21 compares Draco at 6 and 7 bits with INR models using M = 64 and M = 128 at similar or lower storage.
Loading 2112.04267v2…