PaperlayerView exact revision on arXiv

Source-linked AI summary

Goal-Oriented Ordered Autoencoding for Successive Refinement in Progressive Communication

Qi Liao, Dario Bega

arXiv:2610.13426v1cs.IT

TL;DR

Existing methods often prioritize reconstruction, while progressive representations need prefixes directly optimized for task utility under changing bandwidth. GO-OAE learns ordered quantized latents for task inference; on MiniImageNet, it reaches at least 96% of the lossless classification bound at 324 bits per image and outperforms reconstruction-based codecs and uniform-prefix OAEs across budgets.

  • Problem

    Many existing methods remain reconstruction-oriented, and hierarchical representations do not necessarily produce prefixes directly optimized for task utility.

    Source: §I · p. 1, §II · p. 2

  • Method

    GO-OAE jointly learns an ordered quantized latent sequence for task inference, with a weighted conditional information bottleneck objective and an unbiased multi-prefix importance-sampling estimator.

    Source: §I · p. 1

  • Results

    On MiniImageNet, GO-OAE achieves ≥96% of the lossless classification bound at 324 bits/image (683× compression) and outperforms reconstruction-based codecs and uniform-prefix OAEs across budgets.

    Source: §I · p. 2, §VII · p. 9

  • Takeaways & Limitations

    GO-OAE supports task inference from progressively transmitted prefixes, including at extreme compression rates and under variable bandwidth.

    Source: §I · p. 1, §VII · p. 9

Abstract

from arXiv · show

Goal-oriented communication often struggles to sustain downstream task performance under variable and unpredictable bandwidth. We address this problem by proposing GO-OAE (Goal-Oriented Ordered AutoEncoding), which learns a progressively refinable quantized latent sequence, optimized directly for task utility rather than source reconstruction. Each component of the sequence is represented for transmission by its code index, and the learned ordering enables inference from any received prefix of the index sequence, with performance improving as more indices arrive. We prove that optimizing the goal-oriented distortion objective is equivalent to solving a weighted conditional information bottleneck formulation, and introduce an unbiased multi-prefix importance sampling (MPIS) estimator, to reduce GO-OAE training computational cost from O(BK) to O(Bm) with m << K, and reduces the estimator variance by a factor of 1/m. Experiments show that, on MiniImageNet, GO-OAE achieves over 96% of the lossless upper-bound classification accuracy using 324 bits per image (683X compression vs. raw RGB), and already attains over 94% of the upper bound with only 70% of the transmitted indices. It consistently outperforms reconstruction-optimized codecs and ordered autoencoders trained with uniform prefix sampling across all transmission budgets, demonstrating effective communication at extreme compression rates.

I. INTRODUCTION

GO-OAE targets progressive task communication by learning ordered quantized prefixes for downstream inference rather than source reconstruction. On MiniImageNet, it reaches over 96% of lossless classification accuracy at 324 bits per sample and outperforms reconstruction-based codecs and uniform-prefix OAEs across transmission budgets.

II. RELATED WORKS

  • Existing ordered autoencoders improve reconstruction from longer prefixes, but their prefixes are not directly optimized for task utility.Hierarchical representations likewise do not necessarily yield task-optimized ordered latent prefixes.Source: §I · p. 1, §II · p. 2

I. INTRODUCTION

  • GO-OAE jointly optimizes the encoder, vector quantizer, and task decoder so ordered quantized prefixes support downstream inference across variable bandwidths.The encoder maps input components into an ordered latent sequence; transmitted code indices are used directly for task inference.Source: §I · p. 1, Figure 1 · p. 2
  • Its goal-oriented objective is equivalent to a weighted conditional information bottleneck, and MPIS reduces training cost from O(BK) to O(Bm) with estimator variance scaling by 1/m.The objective yields ordering without explicit architectural constraints.Source: §I · p. 1
  • Over 96% of lossless classification accuracy at 324 bits per sample (683× compression), with GO-OAE outperforming reconstruction-based codecs and uniform-prefix OAEs across transmission budgets.The experiments are on MiniImageNet.Source: §I · p. 2

III. DEFINITIONS AND PRELIMINARIES

  • The preliminaries denote an input by x, its latent sequence by z, and a length-k prefix by z≤k; the reconstruction decoder maps the full sequence back to x.The prefix may also be represented as a zero-padded length-K sequence.Source: §III · pp. 2–3

A. Ordered Autoencoder for Reconstruction

The reconstruction-based ordered autoencoder encourages non-increasing incremental information gains across latent components and can be trained by sampling prefix lengths. The paper uses VQ-AE to obtain discrete, progressively transmissible latent codes.

A. Ordered Autoencoder for Reconstruction

  • A reconstruction-based OAE orders z so successive components have non-increasing incremental information gains about the input.The ordered reconstruction objective is designed to achieve this condition.Source: A. Ordered Autoencoder for… · p. 3
  • With a fixed-variance Gaussian decoder and the stated conditional-independence condition, the OAE objective is equivalent up to constants to prefix-averaged mean squared reconstruction loss.An optimal encoder-decoder pair minimizing the objective satisfies the ordered autoencoder condition.Source: A. Ordered Autoencoder for… · p. 3
  • Because the objective averages uniformly over prefix lengths, sampling one length per minibatch gives an unbiased reconstruction-loss estimate.The sampled prefix z≤i is used to optimize the loss.Source: A. Ordered Autoencoder for… · p. 3

B. Vector Quantized Autoencoder

  • VQ-AE maps continuous encoder outputs to learnable codebook embeddings and passes the corresponding discrete indices to the decoder.The indices can be transmitted and mapped back to their embeddings.Source: B. Vector Quantized Autoencoder · p. 3
  • A 512-entry vision codebook represents each latent component index with 9 bits, supporting the paper’s choice of VQ-AE for progressive transmission.The authors also cite more stable, semantically structured representations under progressive truncation as a reason for using discrete latents.Source: B. Vector Quantized Autoencoder · p. 3
  • VQ-AE training combines reconstruction, codebook, and commitment losses; EMA updates codebook embeddings from assigned encoder outputs and improves codebook stability.The commitment loss encourages encoder outputs to commit to quantized latents.Source: B. Vector Quantized Autoencoder · pp. 3–4

IV. GOAL-ORIENTED ORDERED AUTOENCODING

GO-OAE minimizes task loss over all latent-prefix lengths, and its distortion objective is equivalent under stated assumptions to a weighted conditional information bottleneck objective.

IV. GOAL-ORIENTED ORDERED AUTOENCODING

  • GO-OAE trains an encoder–decoder pair to minimize task loss across all prefix lengths.The paper instantiates the task loss as label cross-entropy.Source: §IV · p. 4

A. Connection to Information Bottleneck

  • The GO-OAE distortion function connects task utility across ordered latent prefixes to a weighted conditional information bottleneck formulation.Source: A. Connection to Information Bottleneck · p. 4, §IV · p. 4
  • Under the Markov structure and a Bayes-optimal task decoder, minimizing GO-OAE distortion is equivalent up to a constant to maximizing a weighted conditional information bottleneck objective.The bottleneck term regularizes representation complexity against task performance.Source: A. Connection to Information Bottleneck · p. 4

B. Goal-Oriented Latent Ordering

The GO-OAE objective yields an ordering of task-relevant information under stated assumptions. Its nonuniform prefix weights also motivate a more efficient training estimator than uniform prefix sampling.

B. Goal-Oriented Latent Ordering

  • Under the stated Bayes-optimality and permutation-invariance assumptions, the GO-OAE objective formally characterizes a goal-oriented latent ordering.The resulting ordering has earlier dimensions capture the most critical task-relevant semantics and later dimensions provide finer refinements.Source: B. Goal-Oriented Latent Ordering · p. 4
  • Earlier latent components receive larger effective weights because they appear in more prefixes, yielding monotonically decreasing conditional mutual information across the sequence.The objective's VQ-AE bottleneck, finite codebook, latent dimensionality, and stochastic truncation provide implicit compression.Source: B. Goal-Oriented Latent Ordering · p. 4

V. PROPOSED GO-OAE ALGORITHM

  • GO-OAE assigns nonuniform importance to prefix lengths, so uniform prefix sampling is not an efficient optimization strategy for its objective.Computing every prefix loss costs O(BK) per minibatch, while sampling one prefix can create unbalanced supervision across prefix lengths.Source: §V · p. 5

A. Unbiased Multi-Prefix Importance Sampling Estimator

The multi-prefix importance sampling estimator approximates the GO-OAE objective using sampled prefix lengths while preserving unbiasedness. It reduces computation and estimator variance relative to evaluating all prefixes or relying on a single sampled prefix.

A. Unbiased Multi-Prefix Importance Sampling Estimator

  • Sampling m prefix lengths independently from a distribution q(i) with positive support yields an unbiased stochastic estimator of the GO-OAE objective.The estimator uses the sampled prefix lengths at each minibatch.Source: A. Unbiased Multi-Prefix Importance… · p. 5
  • O(Bm) training complexity replaces O(BK) when m≪K, while stochastic gradient descent optimizes the same objective under the stated interchange assumption.The estimator is unbiased, and its variance decreases by a factor of 1/m.Source: A. Unbiased Multi-Prefix Importance… · p. 5
  • Importance-weighted single-prefix sampling is unbiased but can have high variance when a high-loss prefix has low sampling probability.MPIS reduces estimator variance by a factor of 1/m.Source: A. Unbiased Multi-Prefix Importance… · p. 5
  • The sampling distribution can prioritize earlier latent components through α, with α and m tuned in practice for stable, efficient training.Larger α focuses more on early components with higher importance weights.Source: A. Unbiased Multi-Prefix Importance… · p. 5

B. Overall Training Loss and Algorithm

GO-OAE training combines the stochastic goal-oriented loss with vector-quantization loss and optional regularization. The described training procedure uses semantic and codebook-utilization regularizers and a three-stage schedule.

B. Overall Training Loss and Algorithm

  • The overall GO-OAE loss combines the stochastic goal-oriented loss, vector-quantization loss, and optional regularization.The regularization weight and β control the balance.Source: B. Overall Training Loss and Algorithm · p. 5
  • The optional supervised contrastive regularizer pulls same-class samples closer in the latent space to preserve semantic structure.The method also describes entropy regularization for codebook utilization.Source: B. Overall Training Loss and Algorithm · p. 6
  • Entropy regularization increases the empirical codebook-index entropy to encourage balanced use of codebook embeddings.The regularizer is equivalent to DKL(h∥u), where u is uniform over codebook indices.Source: B. Overall Training Loss and Algorithm · p. 6
  • Training proceeds through full-code pretraining, ordered training with MPIS, and final calibration with occasional full-code supervision.The calibration stage is intended to avoid under-training later components.Source: B. Overall Training Loss and Algorithm · p. 6

VI. EXPERIMENTAL RESULTS

The experiments evaluate GO-OAE on MiniImageNet image classification with a maximum 36-index prefix and a 512-entry codebook. Comparisons vary quantization, task-oriented training, semantic regularization, and prefix sampling.

VI. EXPERIMENTAL RESULTS

  • Full transmission uses 36 code indices from a 512-entry codebook, totaling 324 bits per image and 683× fewer bits than raw RGB storage.The raw RGB baseline is 221,184 bits per image.Source: §VI · p. 6
  • The evaluation compares eight scenarios that share the same backbone and data augmentation but vary quantization, goal-oriented training, SupCon regularization, and prefix sampling.The proposed GO-OAE-I-SC uses MPIS with m=6 prefixes per minibatch.Source: §VI · pp. 6–7
  • The comparison includes a raw-image upper bound, reconstruction-based OAEs, goal-oriented unordered VQ-AEs, and ordered GO-OAE variants.Variants distinguish SupCon use and uniform versus importance-based prefix sampling.Source: §VI · pp. 6–7

A. Datasets

The dataset section specifies MiniImageNet splits, image preparation, and top-1 validation accuracy as the primary task-oriented metric.

A. Datasets

  • MiniImageNet evaluation uses approximately 50k training, 10k validation, and 5k test samples across 100 classes, with every class represented in each split.The data come from the public timm/mini-imagenet split via HuggingFace.Source: A. Datasets · p. 7
  • Images are resized to 96×96 to form 36 spatial patches of 16×16 and normalized to [0, 1].Training augmentation includes random crop, horizontal flip, color jitter, rotation, MixUp, light blurring, and noise corruption.Source: A. Datasets · p. 7
  • Top-1 accuracy on the validation set is the primary goal-oriented metric.Validation uses center resize and ImageNet normalization only.Source: A. Datasets · p. 7

B. Network Architecture and Training Details

GO-OAE is evaluated with CNN and ViT backbones under a shared full-rate budget, with architecture-specific encoders and classification receivers. Training proceeds through full-prefix pretraining and ordered fine-tuning, with prefix calibration also used.

B. Network Architecture and Training Details

  • CNN and ViT experiments share a full-rate budget of 324 bits per image, while the reported discussion focuses on the lighter CNN model.The backbone discussion, including ViT-based insights, is deferred to Section VI-D.Source: B. Network Architecture and Training… · p. 7
  • Both backbones use distinct encoder–quantizer–decoder designs, with about 8M/0.02M/3.3M parameters for CNN and 11M/0.07M/17.9M for ViT.The CNN maps images to a 36×6×6 latent tensor; the ViT uses 36 patch tokens and a perceiver before quantization.Source: B. Network Architecture and Training… · p. 7
  • The default setup uses a 512-entry codebook, 9 bits per component, and K=36, with six sampled prefix lengths per minibatch for MPIS.The CNN and ViT component embedding dimensions are 36 and 128, respectively; SupCon-enabled variants use λSC=3.0.Source: B. Network Architecture and Training… · p. 7
  • Models are trained in stages: full-prefix pretraining, then ordered fine-tuning with either a random single prefix or multiple prefixes.Pretraining uses K=36 for 500 epochs; ordered fine-tuning runs for 300 epochs.Source: B. Network Architecture and Training… · p. 7
  • Calibration mixes full-prefix and truncated-prefix steps to recover full-rate accuracy while preserving low-k performance.The schedule uses 40% full-prefix and 60% truncated-prefix steps for 100 epochs.Source: B. Network Architecture and Training… · p. 7

1) Reconstruction Fidelity vs. Downstream Task Performance:

GO-OAE combines task-oriented ordered representations with semantic regularization and improves classification across transmission budgets. Results also distinguish reconstruction fidelity from task performance and show that progressive refinement depends on the backbone.

1) Reconstruction Fidelity vs. Downstream Task Performance:

  • At K=256, RE-OAE has 18.9% top-1 accuracy versus the 77.5% original-image bound despite satisfying PSNR and visually plausible reconstructions.Task-aware RE-OAE-TA improves performance but remains well below the upper bound.Source: 1) Reconstruction Fidelity vs.… · p. 7

2) Performance Comparison:

  • At k=36, GO-OAE-I-SC reaches 74.4% accuracy versus the 77.5% original-image bound, while GO-OAE-SC improves over GO-OAE by 37.5 percentage points.GO-OAE-I-SC also reaches 72.9% accuracy with k=25 code indices, using 225 bits.Source: 2) Performance Comparison · p. 8, Figure 4 · p. 8

3) Semantic Preservation in Codebook Embeddings - Effect of Supervision:

  • GO-OAE-I-SC maintains stronger semantic structure in the quantized latent space than GO-VQAE, and the EMA codebook update preserves semantic information through quantization.The section attributes gains in unordered and ordered settings to supervised contrastive regularization.Source: 3) Semantic Preservation in Codebook… · p. 8, Figure 5 · p. 8

4) Ordered Latent Space - Effect of Sampling Strategies:

  • GO-OAE-I-SC’s block information gain decays sharply across prefix length, consistent with the predicted ordering of diminishing conditional mutual information.Figure 6 compares GO-OAE-U, GO-OAE-U-SC, and GO-OAE-I-SC across ten prefix blocks.Source: 4) Ordered Latent Space - Effect of… · p. 8, Figure 6 · p. 8

D. Backbone Model Discussion

  • CNN features support complementary latent information and clear progressive refinement, whereas standard ViTs yield nearly flat prefix-performance curves.Using fewer transformer blocks before quantization spreads utility across more components but limits converged accuracy.Source: D. Backbone Model Discussion · pp. 8–9, Figure 7 · p. 8

VII. CONCLUSIONS

  • On MiniImageNet, GO-OAE reaches at least 96% of the lossless bound at 324 bits per image and at least 94% with 70% of the latent components, outperforming reconstruction codecs and uniform-prefix OAEs at all budgets.At k=36, GO-OAE-I-SC reaches 74.4% accuracy (96.1% of the upper bound); at k=25 it reaches 72.9% (94.1% of the upper bound).Source: 2) Performance Comparison · p. 8, Figure 4 · p. 8, §VII · p. 9

APPENDIX

The appendix proofs connect the distortion objective to weighted conditional information gains, derive the optimal ordering of those gains, and establish properties of the MPIS estimator.

A. Proof of Theorem 2

  • Under Bayes-optimal decoders, the distortion objective reduces to conditional entropy terms and is equivalent, up to an additive constant, to maximizing weighted conditional mutual information.The weights are wi=K−i+1, so information from earlier latent components receives greater weight.Source: A. Proof of Theorem 2 · p. 9

B. Proof of Corollary 1

  • At an optimum, incremental task-relevant information gains cannot increase across adjacent latent components, yielding Δ1≥Δ2≥…≥ΔK.The proof uses the strictly decreasing weights and shows that swapping an adjacent inversion would increase the objective.Source: B. Proof of Corollary 1 · p. 9

C. Proof of Corollary 2

  • The MPIS estimator is shown to be unbiased, to have variance reduced by averaging independent samples, and to provide an unbiased gradient under standard regularity conditions.The gradient result assumes interchange of gradient and expectation is permitted.Source: C. Proof of Corollary 2 · pp. 9–10
Loading 2610.13426v1…