PaperlayerView exact revision on arXiv

Source-linked AI summary

Bandwidth-Agile Image Transmission with Deep Joint Source-Channel Coding

David Burth Kurka, Deniz Gündüz

arXiv:2009.12480v2cs.ITcs.LGeess.IVeess.SP

TL;DR

The paper addresses adaptive-bandwidth image transmission over wireless channels when receivers obtain progressively added or independently retrievable layers. It introduces DeepJSCC-l, a deep-learning JSCC family for layered image transmission, and reports practical layered representations with performance comparable to state-of-the-art methods. The approach also supports flexible bandwidth use across the considered transmission settings.

  • Problem

    Practical JSCC schemes for successive refinement and multiple descriptions on realistic information sources and channels are limited, despite the need for flexible layered image transmission.

  • Method

    DeepJSCC-l uses deep-learning autoencoder architectures to encode and decode images over multiple channels for adaptive-bandwidth transmission.

  • Results

    DeepJSCC-l learns layered representations for successive refinement and multiple descriptions with performance comparable to state-of-the-art methods.

  • Takeaways & Limitations

    The learned schemes provide flexible, adaptive-bandwidth image transmission while retaining practical performance across layered transmission modes.

Abstract

from arXiv · show

We propose deep learning based communication methods for adaptive-bandwidth transmission of images over wireless channels. We consider the scenario in which images are transmitted progressively in layers over time or frequency, and such layers can be aggregated by receivers in order to increase the quality of their reconstructions. We investigate two scenarios, one in which the layers are sent sequentially, and incrementally contribute to the refinement of a reconstruction, and another in which the layers are independent and can be retrieved in any order. Those scenarios correspond to the well known problems of \textit{successive refinement} and \textit{multiple descriptions}, respectively, in the context of joint source-channel coding (JSCC). We propose DeepJSCC-$l$, an innovative solution that uses convolutional autoencoders, and present three architectures with different complexity trade-offs. To the best of our knowledge, this is the first practical multiple-description JSCC scheme developed and tested for practical information sources and channels. Numerical results show that DeepJSCC-$l$ can learn to transmit the source progressively with negligible losses in the end-to-end performance compared with a single transmission. Moreover, DeepJSCC-$l$ has comparable performance with state of the art digital progressive transmission schemes in the challenging low signal-to-noise ratio (SNR) and small bandwidth regimes, with the additional advantage of graceful degradation with channel SNR.

I. Introduction

The paper develops practical JSCC for progressively transmitted image layers, addressing successive refinement and multiple descriptions under adaptive bandwidth and uncertain wireless channels. DeepJSCC-l uses deep-learning architectures to learn layered representations while preserving strong image-transmission performance across channel conditions.

  • Motivation: Image layers can be accumulated to improve reconstruction quality, supporting bandwidth-agile communication when bandwidth or transmission urgency is limited.Successive refinement sends a base layer followed by refinement layers, while multiple descriptions allow reconstruction from subsets of independently transmitted layers.
  • Problem: The JSCC multiple-descriptions problem seeks good reconstruction quality for any received subset of noisy channel outputs, and successive refinement is its sequential special case.The JSCC formulation has received less attention than the corresponding source-coding problems.
  • Scope and contribution: The work targets a practical gap because separation-based optimality is established only under ergodic, asymptotic, and unbounded-complexity assumptions.The paper presents its approach as the first practical scheme for successive refinement and multiple-description JSCC on practical sources and channels.
  • Approach: DeepJSCC-l uses convolutional neural-network encoder and decoder functions with the communication channel incorporated as a non-trainable neural-network layer.The approach follows an end-to-end autoencoder design for practical image transmission over wireless channels.
  • Successive refinement: DeepJSCC-l learns successive codewords that refine reconstructions without significant performance losses, and offers three implementations with different time-space complexity trade-offs.Natural images transmitted with DeepJSCC are reported to be nearly successively refinable over Gaussian channels.
  • Multiple descriptions: Independent codewords perform similarly to single-layer transmission when sent separately and improve transmission performance when multiple parts are combined.This extends the learned layered representation from successive refinement to the more general multiple-description problem.
  • Results: The resulting schemes retain graceful degradation, adaptability across AWGN and Rayleigh fading channels, and better or comparable performance to digital schemes.The paper presents these properties as practical advantages for adaptive-bandwidth image communication.

II. System Model

The system transmits an image across parallel noisy channels and reconstructs it from a received subset of channel outputs. Successive refinement uses consecutive subsets, whereas multiple descriptions allow arbitrary combinations, enabling reconstruction with different bandwidth amounts.

  • System model: The system transmits an image x over L parallel channels and forms a reconstruction from the received subset S of channel output vectors.Each channel has a complex input and corresponding complex output vector.
  • Channels: The channel models are AWGN and slow Rayleigh fading, with independent noisy channel realizations for transmitted sequences.AWGN adds circularly symmetric complex Gaussian noise, while slow fading multiplies the signal by a complex Gaussian channel gain.
  • Subset models: Successive refinement uses sequential consecutive channels, while multiple descriptions use arbitrary channel combinations.The two subset structures determine whether layers are accumulated in order or retrieved independently.
  • Bandwidth: Different received-subset sizes provide agile bandwidth, allowing the same image to be transmitted and reconstructed using different bandwidth amounts.The channel bandwidth is k_i, the source bandwidth is n, and k_i/n is the bandwidth ratio for channel i.
  • Evaluation: Performance is evaluated using PSNR between the input image and each reconstruction, with PSNR inversely proportional to MSE.For the considered RGB images, the maximum pixel value MAX is 255.

III. DeepJSCC-l

DeepJSCC-l extends convolutional neural-network-based JSCC to multilayer image transmission using neural encoders and decoders. Its fully convolutional design supports varying image dimensions while maintaining a fixed bandwidth ratio.

  • The channel is included as a differentiable, non-trainable component, and the system can be trained jointly or through sequential greedy optimization.
  • DeepJSCC-l is a family of architectures built from neural encoders and decoders for multilayer transmission.
  • A CNN encoder maps an input image to channel symbols partitioned into layers, while decoders reconstruct images from received layer subsets.
  • The models jointly optimize encoder and decoder parameters using average distortion over partial reconstructions, creating a multi-objective training problem.
  • For images with three color channels, the total bandwidth ratio is k/n = c/48, and each layer's average transmit power is constrained to P = 1.
  • The fully convolutional model accepts arbitrary image dimensions, while experiments evaluate CIFAR-10 images using average test-set PSNR over independent channel realizations.

IV. Successive Refinement JSCC

The successive-refinement model sends a base layer followed by refinement layers, with a single encoder and independent decoders trained jointly to reconstruct from progressively larger prefixes.

  • Successive refinement receives the outputs of the first i channels, with the first channel designated as the base layer and later channels as refinement layers.
  • The primary architecture uses one encoder neural network and L independent decoder networks for progressively available layer subsets.
  • All layers are trained jointly in an autoencoder, with the loss function accounting for reconstructions produced by the different decoders.
  • Experimental results indicate that assigning different weights to decoder distortions has marginal impact on performance.

A. Two-layer Model

The two-layer model learns sequential image representations whose refinement improves reconstruction quality across channel conditions. It also degrades gradually under SNR mismatch and preserves graceful degradation over fading channels.

  • 2 to 3 dB: average PSNR2 is consistently higher than PSNR1 across channel conditions, demonstrating the contribution of the refinement layer.
  • DeepJSCC-l learns a sequential representation in which the input can be reconstructed from the base layer or from both layers.
  • The two-layer system incurs negligible loss from layering compared with equivalent-bandwidth single-transmission baselines.
  • When test SNR is below the training SNR, both reconstructions deteriorate gradually rather than abruptly, indicating robustness to SNR mismatch.
  • Performance improves gradually with channel SNR, so DeepJSCC-l exhibits graceful degradation instead of the digital cliff effect.
  • Over slow Rayleigh fading, PSNR is lower than with AWGN because of channel uncertainty, but graceful degradation and limited loss relative to the single-layer baseline remain.
  • DeepJSCC-l adapts to channel uncertainty without explicit pilot signals or channel estimation, including when trained over fading channels.
  • Because the reconstruction after the first l layers depends on the random channel realization, the refinement layer cannot simply transmit residual information known in advance.

C. Multiple Layers

With more layers, DeepJSCC-l increases image quality at each progressive step while preserving the performance of earlier layers. The resulting layer interactions are nearly independent, including at constrained low SNR and bandwidth.

  • For L = 5 layers, each transmitted at k_i/n = 1/12, the performance gain from each refinement layer gradually decreases.
  • Adding refinement layers increases overall image quality at every step, although successive improvements diminish as lower layers transmit the main image features.
  • Adding layers during training has low impact on the performance of the first layers, and comparisons across L ∈ {2, 3, 4, 5} show negligible loss from new layers.
  • The stability of early-layer performance suggests near-independence between layers despite the first-layer code being shared across the model.
  • At SNR = 19dB, DeepJSCC-l achieves superior first-layer performance to separation-based schemes using JPEG2000 or BPG with ideal capacity-achieving channel coding.

D. Comparison with Digital Transmission

DeepJSCC-l is compared with layered JPEG2000 and BPG transmission across different layer counts, SNRs, and bandwidth constraints. It is superior or comparable, especially for lower layers and difficult low-SNR, bandwidth-limited conditions.

  • Progressive transmission can divide fixed bandwidth into as many as L = 16 layers, allowing receivers to stop once a target quality is reached.
  • Digital baselines use JPEG2000 or BPG source coding followed by an ideal capacity-achieving channel code, serving as an upper bound for separation-based schemes.
  • DeepJSCC-l achieves performance superior to or comparable with JPEG2000 and BPG, particularly for lower layers.
  • At SNR 1 dB and constrained bandwidth, JPEG2000 and BPG cannot compress images to the channel-supported level, producing flat performance curves.
  • ImageNet-trained extended models are evaluated on Kodak, with refined reconstructions improving visible details over base reconstructions.

E. Alternative Architectures

The paper presents alternative successive-refinement architectures that trade memory, training time, inference overhead, and extensibility rather than performance. Single-decoder and residual designs achieve performance close to the single-transmission reference, while residual transmission supports later layer additions.

  • The alternative architectures provide comparable performance, with trade-offs primarily involving space and time complexity.
  • Single Decoder: A single decoder exposed to different code lengths adapts to available bandwidth while using one encoder and decoder for all layers.
  • Single Decoder: Randomly varying transmitted-layer length during training is essential for preserving performance across layers; sequential subset training harms early layers.
  • Single Decoder: The single-decoder scheme performs close to the single-transmission bound and matches the multiple-decoder architecture while reducing memory and processing requirements.
  • Residual Transmission: Residual transmission encodes estimated missing information and combines each residual refinement with the previous reconstruction.
  • Residual Transmission: With m = 10 channel realizations, residual transmission achieves results close to the other schemes, with its first layer matching single transmission at k/n = 1/12.
  • Residual Transmission: Residual transmission is computationally more demanding but allows new refinement layers without retraining earlier networks.

3) Architecture Comparison:

The three successive-refinement architectures achieve nearly the same performance on an AWGN channel, so practical selection depends mainly on memory, training, inference, and extensibility requirements.

  • Multiple decoders, a single decoder, and residual transmission all achieve nearly the same performance for successively refinable image representations over AWGN.
  • The single-decoder architecture minimizes memory by using one encoder and decoder regardless of L.
  • Residual transmission has extra inference overhead from repeated transmission emulation and output mixing.
  • Multiple decoders train layers simultaneously and in parallel, whereas single-decoder and residual schemes require sequential training.
  • Residual transmission is the only scheme that can add new layers afterward without retraining previous networks.

V. Multiple Description JSCC

Multiple-description JSCC must reconstruct images from arbitrary subsets of independently transmitted layers, making it more challenging than successive refinement. DeepJSCC-l learns complementary descriptions that improve joint reconstructions, although combined performance remains below a joint-only target.

  • For L layers, the architecture uses virtual decoders for all nonempty subsets, totaling 2^L − 1 possible decoders.
  • Multiple-description coding generalizes successive refinement and is more challenging because every subset must support reconstruction.
  • Multiple-description coding supports independent reception of arbitrary channel subsets, unlike successive refinement, which assumes an ordered sequence.
  • Single-layer reconstructions are equivalent across decoders and nearly match a single-layer encoder with the same bandwidth.
  • With multiple layers, joint decoding improves quality over single-layer decoding but remains inferior to a scheme targeting only the joint decoder.
  • Unlike successive refinement, multiple descriptions incur a performance loss because each single-layer receiver must reconstruct the whole image independently.
  • DeepJSCC-l outperforms repetition coding at all SNR values, indicating that independent layers learn complementary representations rather than repeating one codeword.
  • ImageNet-trained extensions evaluated on Kodak show additional visual detail in refined reconstructions, including a clearer letter “A.”

B. Single Encoder-Decoder Network

The single encoder-decoder architecture supports multiple-description transmission with one decoder handling different received-layer subsets. Its performance is somewhat below multiple decoders but remains comparable to equivalent single transmissions, while DeepJSCC-l supports layered transmission across settings.

  • Architecture: A single decoder is trained to reconstruct images from different subsets of received layers.The number of possible subsets grows exponentially with L, increasing learning complexity.
  • Performance: For L = 2 and L = 3, single-layer transmission has performance comparable to a single transmission with equivalent bandwidth.The single-decoder results are not as good as those of multiple decoders.
  • Performance: Multiple layers generally outperform repetition schemes at the same bandwidth.
  • Conclusions: DeepJSCC-l learns layered representations for successive refinement and multiple-description transmission.
  • Conclusions: Across a wide range of SNRs and channel bandwidths, performance is comparable to state-of-the-art methods.
  • Conclusions: The model degrades gracefully under channel-quality mismatch and can operate over diverse channels, including fading channels.

Appendix A

The appendix studies how loss-function weights shape the trade-offs among reconstructions from different layer subsets. It shows that weighting can prioritize side or joint decoders, while intermediate choices preserve overall performance more effectively.

  • Loss weighting: The successive-refinement and multiple-description objectives are formulated as multi-objective losses with adjustable weights.Equal weights are used in the reported experiments, but alternative weights can prioritize reconstructions with different bandwidths.
  • Successive Refinement Trade-offs: For L = 2, extreme λ1 values make one layer dominate while the other loses about 12.5 dB; intermediate values have small overall impact.
  • Multiple Description Trade-offs: The multiple-description objective balances quality from individual layers against quality from jointly decoding all layers.The appendix simplifies the targets to single-layer and all-layer reconstructions.
  • Multiple Description Trade-offs: As α1 decreases, average side-decoder performance improves toward single-transmission performance; as α1 increases, joint-decoder performance improves.The trade-off is between independently decoded descriptions and their combined reconstruction.
  • Multiple Description Trade-offs: When α1 approaches 1, performance approaches that of a single decoder using all available channel bandwidth.
  • Multiple Description Trade-offs: Increasing α2 emphasizes the second descriptor alone and reduces the performance of the two descriptors combined.The appropriate balance depends on the application and the likelihood of different received subsets.
  • Multiple Description Trade-offs: The suitable weighting balance depends on the application and the likelihood of experiencing particular subsets of layers.
Loading 2009.12480v2…