PaperlayerView exact revision on arXiv

Source-linked AI summary

FRAPPE: Full Input, Residual Output Autoencoding with Projection Pursuit Encoder

Dan Jacobellis, Neeraja J. Yadwadkar

arXiv:2605.28992v1eess.IV

TL;DR

Existing codecs face a rate-distortion-complexity plateau, while learned codecs struggle with rate adaptation and affordable real-time encoding. FRAPPE introduces a projection-pursuit residual autoencoder with naturally ordered latent channels, and FRAPPE-Image performs favorably in the rate-distortion-encoding-complexity trade-off against existing codecs.

  • Problem

    Existing codecs and learned autoencoders remain limited by encoding cost, rate adaptation, or real-time operation on commodity, resource-constrained hardware.

  • Method

    FRAPPE uses full-input projection-pursuit residual autoencoding to order latent channels by importance for zero-overhead variable-rate and progressive coding.

  • Results

    FRAPPE-Image achieves 74–168 MPx/s encoding throughput, while mbt2018 is up to ∼1000× slower at matched rates.

  • Takeaways & Limitations

    FRAPPE-Image provides a practical image-compression system with a favorable rate-distortion-encoding-complexity trade-off for resource-constrained sensors.

  • Takeaways & Limitations

    Experiments cover only RGB images, despite the framework’s stated applicability to other signal types.

Abstract

from arXiv · show

Media compression standards have reached a plateau in terms of the rate-distortion-complexity trade-off, limiting the ability to offload expensive AI perception to the cloud in applications like robotics, wearables, and remote sensing. DNN-based codecs improve compression efficiency, but at a cost: they cannot easily adapt to large changes in available bitrate, and real-time encoding requires expensive, power-hungry GPUs that prohibit use on low-cost or resource-constrained platforms. To address these limitations, we propose a novel autoencoding framework (FRAPPE) that uses the Full input to predict the Residual output via a Projection Pursuit Encoder. FRAPPE's encoding objective naturally sorts latent channels by importance, allowing zero-overhead variable-rate coding. Unlike RNN-based learned codecs, whose encoder consumes the previous reconstruction's residual, or RVQ-style codecs, whose codebooks must be applied sequentially, FRAPPE's analysis path is an embarrassingly parallel DAG of independent input projections. Using FRAPPE, we build a variable-rate RGB image codec (FRAPPE-Image), and evaluate its rate-distortion-complexity trade-off against standard image codecs. At high compression ratios (approx. 0.1 bpp) FRAPPE-Image provides higher perceptual quality than AVIF with 47 times faster encoding, making it capable of real-time 1080p, 30fps CPU-only encoding. Our code and pre-trained models are available: https://github.com/UT-SysML/FRAPPE .

I. INTRODUCTION

Existing codecs face a rate-distortion-complexity plateau, while learned autoencoders struggle with rate adaptation, encoding cost, and commodity-hardware real-time operation. FRAPPE addresses these constraints with projection-pursuit residual autoencoding that enables variable-rate progressive coding and resource-efficient encoding.

  • Learned autoencoders compromise on rate adaptation, encoding cost, or real-time commodity-hardware operation, despite conventional codecs’ rising complexity and power consumption,,.
  • FRAPPE uses the full input to predict the residual output through a Projection-Pursuit Encoder, sorting latent channels by importance for zero-overhead variable-rate and progressive coding.A single set of encoder weights supports these coding modes.
  • The proposed framework targets competitive high-compression rate-distortion performance and low encoding cost for resource-constrained sensors, while supporting variable-rate and progressive compression.
  • The authors instantiate a practical image compression system and evaluate FRAPPE-Image against conventional and learned codecs for rate-distortion-complexity trade-offs.
  • Background and related work: Unlike RNN-based learned codecs [5], and residual-vector-quantized neural codecs –, FRAPPE avoids sequential encoding dependencies through its projection-pursuit design.RNN-based codecs evaluate the decoder inside the encoding loop, while residual vector quantization retains a sequential quantizer chain.

II. PROPOSED METHOD

FRAPPE uses independently projected, progressively ordered latent channels to provide variable-rate coding with a lightweight, parallelizable encoder. Its multiscale analysis path quantizes channels independently, while a merged synthesis transform reconstructs outputs from adapted latents.

  • Design goals: FRAPPE targets zero-overhead variable-rate coding, standardized-codec-competitive rate–distortion performance, and high-throughput encoding on low-power sensors without GPUs or accelerators.A single set of encoder weights supports progressive coding by transmitting different channel counts.
  • Analysis transform: Each analysis channel independently applies a learned linear projection to a non-overlapping patch, then undergoes signed 8-bit companding, quantization, and independent per-scale entropy coding.Scale groups use distinct patch sizes and latent-channel counts; adapted scale tensors are concatenated before synthesis.
  • Progressive channel ordering: Progressive residual prediction sorts latent channels by importance because each new channel captures variance unexplained by its predecessors, without an explicit decorrelation loss.The training target is the output-space residual, while the encoder still receives the full input; the schedule relaxes patch size and λm monotonically.
  • Encoder and decoder: The analysis encoder consolidates exactly into five independent strided-convolution layers for FRAPPE-Image, while the merged synthesis network uses channel-count-specific decoder snapshots.The synthesis stack uses width 768 and twelve ConvNeXt-style residual blocks in FRAPPE-Image.
  • Computational structure: FRAPPE-Image’s analysis path costs approximately 68 ops/pixel and forms an unconstrained DAG whose scale projections can be pipelined or evaluated in parallel.The encoder has no recurrent dependency [5], or sequential residual-quantizer chain –; the finest scale adds only 3 ops/pixel.

III. EXPERIMENTAL DATA AND RESULTS

On Kodak, FRAPPE-Image is evaluated against conventional and neural codecs for rate-distortion-complexity using CPU-only encoding. Although mbt2018 achieves higher PSNR in the low-rate regime, it is far slower than FRAPPE-Image, limiting its use for asymmetric on-sensor encoding.

  • Experimental setup: The Kodak evaluation compares FRAPPE-Image with JPEG, AVIF, mbt2018, and WaLLoC using bpp rate and PSNR, SSIM, and DISTS distortion metrics.Encoding throughput is measured end-to-end on an AMD EPYC 9354 CPU, without GPUs or hardware accelerators at inference.
  • Rate-distortion-complexity results: 74–168 MPx/s versus 0.16–0.17 MPx/s gives FRAPPE-Image up to ∼1000× higher encoding throughput than mbt2018 at matched rates.The comparison uses the same CPU testbed and motivates FRAPPE for asymmetric, on-sensor encoding.
  • Rate-distortion-complexity results: mbt2018 retains a +2.2 to +4.2 dB BD-PSNR advantage over FRAPPE-Image across the [0.1, 1) bpp band.Its PSNR-optimal regime is therefore impractical for the asymmetric encoding setting targeted by FRAPPE.
  • Experimental setup: Throughput is reported as the median of five timed CPU runs, including analysis, companding/quantization, and JPEG-LS entropy coding.AVIF uses Pillow over libavif at default speed and effort, without tile or…

IV. CONCLUSION

The paper presents FRAPPE for zero-overhead variable-rate lossy compression on resource-constrained sensors and instantiates it as FRAPPE-Image. FRAPPE-Image compares favorably with existing codecs in rate–distortion–encoding-complexity trade-offs, while the framework applies to 1D, 2D, and 3D signals with arbitrary channel counts.

  • FRAPPE enables zero-overhead variable-rate lossy compression for resource-constrained sensors.
  • FRAPPE-Image provides a practical image compression system with a favorable rate–distortion–encoding-complexity trade-off against existing codecs.
  • The framework applies to 1D, 2D, and 3D signals with arbitrary channel counts.

APPENDIX A REGIME-LOCALIZED BJONTEGAARD-DELTA ANALYSIS

The appendix compares FRAPPE-Image with three additional CPU-only baselines using representative operating points across rate- and quality-localized regimes. It reports matched-rate BD-Metric differences and matched-quality BD-Rate differences relative to FRAPPE.

  • Comparison setup: Tables I and II compare FRAPPE-Image with JPEG XL, LiVeAction, and MCUCoder using one representative point per codec–regime pair.MCUCoder throughput uses fp32 PyTorch and is therefore an upper bound on its deployed INT8/CMSIS-NN encoder.
  • Rate-localized analysis: Table I bins operating points by rate and reports BD-PSNR, BD-SSIM, and BD-DISTS as average distortion differences at matched rate.Rates use 1/3-decade bpp regimes above 0.0464 bpp, while lower-rate points are collapsed into a single < 0.0464 regime; positive values favor the test codec over FRAPPE.
  • Quality-localized analysis: Table II bins operating points by PSNR quality regimes and reports BD-Rate as the average percentage rate difference at matched quality.The PSNR regimes span 22.5 to 32.5 dB in 2.5 dB increments.

APPENDIX B EVALUATION METHODOLOGY DETAILS

The appendix clarifies harness choices that affect reported rate-distortion and throughput numbers, including different input resolutions and an mbt2018 likelihood-based bitrate measurement. It also defines how Kodak BD-metric tables compare codecs against FRAPPE-Image across rate and distortion regimes.

  • Throughput vs rate-distortion input shape: Rate-distortion metrics use native Kodak resolutions, while throughput uses 512×512 center crops to standardize denominators and avoid codec divisibility constraints.The mbt2018 baseline requires dimensions that are multiples of 64.
  • mbt2018 bitstream: mbt2018 reports likelihood-based bpp without invoking autoregressive bitstream compression, so its plotted CPU throughput excludes serialization time.The likelihood-based bpp is a tight lower bound, while real CPU encoding would be substantially slower because autoregressive serialization dominates.
  • TABLE I RATE-BINNED BD-METRIC ON KODAK: Table I reports rate-binned BD-PSNR, BD-SSIM, and BD-DISTS differences versus FRAPPE-Image, with positive values indicating higher test-codec quality.The comparisons are computed at matched bitrate on Kodak.
  • TABLE II PSNR-BINNED BD-RATE ON KODAK: Table II reports PSNR-binned BD-Rate differences versus FRAPPE-Image, where negative values indicate that the test codec uses less rate at matched quality.The PSNR column defines the regime; SSIM/DISTS cells may be unavailable when their metrics do not overlap despite matched PSNR.
Loading 2605.28992v1…