Source-linked AI summary
COIN: COmpression with Implicit Neural representations
Emilien Dupont, Adam Goliński, Milad Alizadeh, Yee Whye Teh, Arnaud Doucet
TL;DR
Image compression often relies on autoencoders, latent codes, and learned distributions, while COIN instead fits a small MLP mapping pixel locations to RGB values and stores its quantized weights. Experiments show that COIN can outperform JPEG at low bit-rates without entropy coding, although encoding is slow and the method remains behind state-of-the-art compression methods.
Problem
Neural image compression typically uses autoencoders with discretized latent codes and learned distributions, whereas COIN investigates storing an overfit image-specific network instead.
Method
COIN fits an MLP mapping pixel locations to RGB values, quantizes its weights, and reconstructs the image by evaluating the network at pixel locations.
Results
COIN outperforms JPEG at low bit-rates without entropy coding or learning a distribution over weights.
Takeaways & Limitations
COIN demonstrates a simple model-based image compression approach with progressive decoding and much smaller decoder-side memory requirements than other neural compression baselines.
Takeaways & Limitations
Encoding is slow because each image requires solving an optimization problem, and the method performs worse than state-of-the-art compression methods.
Abstract
from arXiv · showhide
We propose a new simple approach for image compression: instead of storing the RGB values for each pixel of an image, we store the weights of a neural network overfitted to the image. Specifically, to encode an image, we fit it with an MLP which maps pixel locations to RGB values. We then quantize and store the weights of this MLP as a code for the image. To decode the image, we simply evaluate the MLP at every pixel location. We found that this simple approach outperforms JPEG at low bit-rates, even without entropy coding or learning a distribution over weights. While our framework is not yet competitive with state of the art compression methods, we show that it has various attractive properties which could make it a viable alternative to other neural data compression approaches.
1 INTRODUCTION
COIN compresses images by fitting a small implicit neural representation that maps pixel locations to RGB values, then transmitting quantized network weights. The approach outperforms JPEG at low bit-rates without entropy coding or learned weight distributions.
- COIN fits an image with an MLP mapping pixel locations to RGB values and uses its quantized weights as the image code.The network weights are transmitted and later evaluated to reconstruct the image.
- Sine-activated MLPs can fit 393k-pixel images with surprisingly small networks containing 8k parameters.
- At low bit-rates, COIN outperforms JPEG without entropy coding or learning a distribution over weights.
- COIN supports progressive decoding by evaluating the image function at progressively higher resolutions.
2 METHOD
COIN converts image compression into model compression by fitting a coordinate-to-RGB function, restricting and quantizing its parameters, and evaluating the resulting network during decoding.
- Encoding overfits an MLP to the image, quantizes its weights, and transmits them; decoding evaluates the MLP at all pixel locations.
- 2.1 ENCODING: The image is represented by a function fθ mapping pixel locations to RGB values and fitted using mean squared error.
- Sine activations empirically yield better results than Fourier features for a given parameter budget.
- 2.1 ENCODING: Restricting the number of MLP weights trades off distortion against rate and casts data compression as model compression.
- Weights are reduced from 32-bit to 16-bit precision, while architecture search varies the MLP width and number of layers.
- Progressive decoding can reconstruct image parts or low-resolution images first by evaluating the function at selected pixel locations.
3 RELATED WORK
COIN relates implicit neural representations and neural data compression by using per-image model fitting, while explicitly framing image compression as model compression.
- Implicit neural representations: Implicit neural representations compactly encode high-resolution signals, and COIN extends this idea by using carefully chosen architectures and quantized weights.
- Neural data compression: Neural image compression commonly uses hierarchical variational autoencoders with learned priors and discretized latent variables for entropy coding.
- Neural data compression: Several methods use per-instance optimization to reduce amortization or discretization gaps in latent-variable compression.
- Model compression: COIN explicitly casts data compression as model compression, connecting it to a broader model-compression literature.
4 EXPERIMENTS
Experiments evaluate COIN on Kodak images against neural and traditional codecs, finding low-rate improvements over JPEG and substantially smaller decoder-side models. Training dynamics and architecture choice materially affect compression quality.
- Evaluation setup: Experiments use the 24-image Kodak dataset and compare COIN with three neural baselines plus JPEG, JPEG2000, BPG, and VTM.The images are 768×512, and evaluation uses CompressAI models alongside a PyTorch COIN implementation.
- Rate-distortion performance: At low bit-rates, COIN improves upon JPEG without entropy coding.The authors describe this result as promising, while noting that the method remains behind state-of-the-art compression methods.
- Model size: At 0.3bpp, COIN requires 14kB, compared with 10MB to 40MB for other baselines.This comparison concerns decoder-side model memory requirements.
- Encoding optimization dynamics: COIN outperforms JPEG after 15k iterations on the illustrated training example and continues improving afterward.Because optimization is noisy, the experiment retains the model with the best PSNR.
- Architecture choice: Compression quality depends on architecture choice, with different optimal architectures at different bpp values.Figure 5 compares valid networks with the same 0.3bpp size.
5 SCOPE, LIMITATIONS AND FUTURE WORK
COIN’s main scope boundaries are slow per-image encoding and weaker performance than state-of-the-art compression methods. Future work targets faster encoding, learned weight distributions, and improved architectures.
- Limitations: Encoding is slow because each image requires solving an optimization problem.The authors note that this upfront cost can fit one-to-many media distribution, and suggest meta-learning or amortized inference as possible remedies.
- Limitations: COIN performs worse than state-of-the-art compression methods.The paper identifies reducing this performance gap as a promising direction.
- Future work: Future work includes learning distributions over function weights to seek compression gains.The motivation comes from recent generative modeling work on implicit representations.
- Future work: Future work also includes meta-learning or amortization for faster encoding and architecture refinement through search or pruning.These directions address encoding cost and the design of image-representing functions.
6 CONCLUSION
The conclusion presents COIN as image compression by fitting neural networks to pixels and storing their weights. Experiments show low-rate improvement over JPEG without entropy coding, motivating further work on neural data compression.
- Conclusion: COIN compresses images by fitting neural networks to pixels and storing the resulting model weights.The conclusion frames this as a new method for image compression.
- Conclusion: Experiments show that COIN can outperform JPEG at low bit-rates without entropy coding.The authors present this as evidence supporting further research on neural data compression.
A EXPERIMENTAL DETAILS
The experiments use coordinate-to-RGB MLPs with sine activations, normalized inputs and outputs, and bpp-specific depth and width settings. Training uses Adam for 50k iterations with 16-bit-oriented architecture budgets described elsewhere.
- Model and normalization: Models use two input dimensions for (x, y) coordinates and three output dimensions for RGB values.Coordinates are normalized to [−1, 1], while RGB values are normalized to [0, 1].
- Training: Sine nonlinearities are used at every layer except the last, with the initialization described in [30].The learning rate is 2e-4, and training uses Adam for 50k iterations.
- Architectures: The 0.07bpp configuration uses 5 layers of width 20.
- Architectures: The 0.15bpp configuration uses 5 layers of width 30.
- Architectures: The 0.3bpp configuration uses 10 layers of width 28, while 0.6bpp uses 10 layers of width 40.
- Architectures: The 1.2bpp configuration uses 13 layers of width 49.Code for reproducing the experiments is provided in the referenced repository.
B ADDITIONAL RESULTS
Across the Kodak dataset at 0.3bpp, COIN and JPEG show closely matched distortion patterns: images difficult for one codec are also difficult for the other.
- At 0.3bpp, COIN and JPEG have closely following distortion values across all Kodak images.
- Images that are difficult for COIN to encode are also difficult for JPEG to encode.
- Figure 6 plots the performance of both codecs for every image in the Kodak dataset.
C QUALITATIVE RESULTS
The qualitative results compare COIN and JPEG reconstructions across Kodak images and bit-rates, with PSNR values reported for each comparison.
- Qualitative results compare compression artifacts from COIN and JPEG on the Kodak dataset.
- At 0.15bpp on image 2, COIN reaches 28.69dB PSNR versus JPEG’s 24.67dB.
- At 0.15bpp on image 3, COIN reaches 29.02dB PSNR versus JPEG’s 23.63dB.
- At 0.15bpp on image 15, COIN reaches 27.35dB PSNR versus JPEG’s 21.74dB.
- At 0.3bpp, COIN and JPEG are close on image 15, with PSNRs of 29.31dB and 28.85dB respectively.
- At 0.15bpp on image 16, COIN reaches 27.19dB PSNR versus JPEG’s 24.16dB.