FRAPPE: Full Input, Residual Output Autoencoding with Projection Pursuit Encoder

• Learned autoencoders compromise on rate adaptation, encoding cost, or real-time commodity-hardware operation, despite conventional codecs’ rising complexity and power consumption,,.Source: §I · p. 1
• FRAPPE uses the full input to predict the residual output through a Projection-Pursuit Encoder, sorting latent channels by importance for zero-overhead variable-rate and progressive coding.A single set of encoder weights supports these coding modes.Source: §I · p. 1
• The proposed framework targets competitive high-compression rate-distortion performance and low encoding cost for resource-constrained sensors, while supporting variable-rate and progressive compression.Source: §I · p. 1
• The authors instantiate a practical image compression system and evaluate FRAPPE-Image against conventional and learned codecs for rate-distortion-complexity trade-offs.Source: §I · p. 1

Background and related work

• Unlike RNN-based learned codecs [5], and residual-vector-quantized neural codecs –, FRAPPE avoids sequential encoding dependencies through its projection-pursuit design.RNN-based codecs evaluate the decoder inside the encoding loop, while residual vector quantization retains a sequential quantizer chain.Source: §I · pp. 1–2

Design goals

• FRAPPE targets zero-overhead variable-rate coding, standardized-codec-competitive rate–distortion performance, and high-throughput encoding on low-power sensors without GPUs or accelerators.A single set of encoder weights supports progressive coding by transmitting different channel counts.Source: §II · p. 2, Figure 2 · p. 4

Analysis transform

• Each analysis channel independently applies a learned linear projection to a non-overlapping patch, then undergoes signed 8-bit companding, quantization, and independent per-scale entropy coding.Scale groups use distinct patch sizes and latent-channel counts; adapted scale tensors are concatenated before synthesis.Source: §II · p. 2

Progressive channel ordering

• Progressive residual prediction sorts latent channels by importance because each new channel captures variance unexplained by its predecessors, without an explicit decorrelation loss.The training target is the output-space residual, while the encoder still receives the full input; the schedule relaxes patch size and λm monotonically.Source: §II · pp. 2–3

Encoder and decoder

• The analysis encoder consolidates exactly into five independent strided-convolution layers for FRAPPE-Image, while the merged synthesis network uses channel-count-specific decoder snapshots.The synthesis stack uses width 768 and twelve ConvNeXt-style residual blocks in FRAPPE-Image.Source: §II · p. 3

Computational structure

• FRAPPE-Image’s analysis path costs approximately 68 ops/pixel and forms an unconstrained DAG whose scale projections can be pipelined or evaluated in parallel.The encoder has no recurrent dependency [5], or sequential residual-quantizer chain –; the finest scale adds only 3 ops/pixel.Source: §II · p. 4

Experimental setup

• The Kodak evaluation compares FRAPPE-Image with JPEG, AVIF, mbt2018, and WaLLoC using bpp rate and PSNR, SSIM, and DISTS distortion metrics.Encoding throughput is measured end-to-end on an AMD EPYC 9354 CPU, without GPUs or hardware accelerators at inference.Source: §III · pp. 4–5

Rate-distortion-complexity results

• 74–168 MPx/s versus 0.16–0.17 MPx/s gives FRAPPE-Image up to ∼1000× higher encoding throughput than mbt2018 at matched rates.The comparison uses the same CPU testbed and motivates FRAPPE for asymmetric, on-sensor encoding.Source: Figure 3 · p. 5, §III · p. 5
• mbt2018 retains a +2.2 to +4.2 dB BD-PSNR advantage over FRAPPE-Image across the [0.1, 1) bpp band.Its PSNR-optimal regime is therefore impractical for the asymmetric encoding setting targeted by FRAPPE.Source: §III · p. 5

Experimental setup

• Throughput is reported as the median of five timed CPU runs, including analysis, companding/quantization, and JPEG-LS entropy coding.AVIF uses Pillow over libavif at default speed and effort, without tile or…Source: §III · p. 5

• FRAPPE enables zero-overhead variable-rate lossy compression for resource-constrained sensors.Source: §IV · p. 5
• FRAPPE-Image provides a practical image compression system with a favorable rate–distortion–encoding-complexity trade-off against existing codecs.Source: §IV · p. 5
• The framework applies to 1D, 2D, and 3D signals with arbitrary channel counts.Source: §IV · p. 5

Comparison setup

• Tables I and II compare FRAPPE-Image with JPEG XL, LiVeAction, and MCUCoder using one representative point per codec–regime pair.MCUCoder throughput uses fp32 PyTorch and is therefore an upper bound on its deployed INT8/CMSIS-NN encoder.Source: Appendix A · p. 7

Rate-localized analysis

• Table I bins operating points by rate and reports BD-PSNR, BD-SSIM, and BD-DISTS as average distortion differences at matched rate.Rates use 1/3-decade bpp regimes above 0.0464 bpp, while lower-rate points are collapsed into a single < 0.0464 regime; positive values favor the test codec over FRAPPE.Source: Appendix A · p. 7

Quality-localized analysis

• Table II bins operating points by PSNR quality regimes and reports BD-Rate as the average percentage rate difference at matched quality.The PSNR regimes span 22.5 to 32.5 dB in 2.5 dB increments.Source: Appendix A · p. 7

Throughput vs rate-distortion input shape

• Rate-distortion metrics use native Kodak resolutions, while throughput uses 512×512 center crops to standardize denominators and avoid codec divisibility constraints.The mbt2018 baseline requires dimensions that are multiples of 64.Source: Appendix B · p. 7

mbt2018 bitstream

• mbt2018 reports likelihood-based bpp without invoking autoregressive bitstream compression, so its plotted CPU throughput excludes serialization time.The likelihood-based bpp is a tight lower bound, while real CPU encoding would be substantially slower because autoregressive serialization dominates.Source: Appendix B · p. 7

TABLE I RATE-BINNED BD-METRIC ON KODAK

• Table I reports rate-binned BD-PSNR, BD-SSIM, and BD-DISTS differences versus FRAPPE-Image, with positive values indicating higher test-codec quality.The comparisons are computed at matched bitrate on Kodak.Source: Appendix B · p. 8, Table I · p. 8

TABLE II PSNR-BINNED BD-RATE ON KODAK

• Table II reports PSNR-binned BD-Rate differences versus FRAPPE-Image, where negative values indicate that the test codec uses less rate at matched quality.The PSNR column defines the regime; SSIM/DISTS cells may be unavailable when their metrics do not overlap despite matched PSNR.Source: Appendix B · p. 8, Table II · p. 8