PaperlayerView exact revision on arXiv

Source-linked AI summary

DeepJSCC-f: Deep Joint Source-Channel Coding of Images with Feedback

David Burth Kurka, Deniz Gündüz

arXiv:1911.11174v2cs.ITcs.LGeess.IVeess.SPstat.ML

TL;DR

The paper addresses the lack of practical JSCC schemes that exploit channel output feedback despite separation’s finite-blocklength limitations. It proposes the autoencoder-based DeepJSCC-f for image transmission with perfect or noisy feedback, and reports considerable gains in reconstruction quality or average delay.

  • Problem

    Practical JSCC schemes were not known to exploit channel output feedback and surpass separation-based schemes despite finite-blocklength limitations.

  • Method

    DeepJSCC-f uses an autoencoder architecture for image JSCC over an AWGN channel with perfect or noisy channel output feedback.

  • Results

    Experiments show considerable performance gains over JSCC without feedback, practical codec-plus-channel-code schemes, and ideal capacity-achieving channel codes.

  • Takeaways & Limitations

    Using feedback supports a multi-step image-transmission strategy that improves transmission quality.

Abstract

from arXiv · show

We consider wireless transmission of images in the presence of channel output feedback. From a Shannon theoretic perspective feedback does not improve the asymptotic end-to-end performance, and separate source coding followed by capacity-achieving channel coding, which ignores the feedback signal, achieves the optimal performance. It is well known that separation is not optimal in the practical finite blocklength regime; however, there are no known practical joint source-channel coding (JSCC) schemes that can exploit the feedback signal and surpass the performance of separation-based schemes. Inspired by the recent success of deep learning methods for JSCC, we investigate how noiseless or noisy channel output feedback can be incorporated into the transmission system to improve the reconstruction quality at the receiver. We introduce an autoencoder-based JSCC scheme, which we call DeepJSCC-f, that exploits the channel output feedback, and provides considerable improvements in terms of the end-to-end reconstruction quality for fixed-length transmission, or in terms of the average delay for variable-length transmission. To the best of our knowledge, this is the first practical JSCC scheme that can fully exploit channel output feedback, demonstrating yet another setting in which modern machine learning techniques can enable the design of new and efficient communication methods that surpass the performance of traditional structured coding-based designs.

I. Introduction

Separation is optimal asymptotically, but practical finite-blocklength systems lack JSCC schemes that exploit feedback effectively. The paper introduces DeepJSCC-f, an autoencoder-based image-transmission architecture using noisy or noiseless channel output feedback.

  • Shannon separation is asymptotically optimal, but its practical optimality requires unlimited delay and complexity.
  • Practical JSCC research has mostly studied idealized distributions or optimized components within inherently separate designs.
  • Feedback does not increase memoryless-channel capacity, so separation remains theoretically optimal for source-channel coding with feedback.
  • The paper targets practical JSCC schemes that exploit noisy or noiseless channel output feedback using deep neural networks.
  • DeepJSCC-f is presented as a first practical joint design whose source-to-channel mapping depends on feedback and surpasses separate schemes at finite blocklength.
  • Feedback improves DeepJSCC-f performance, including reconstruction quality for fixed-length transmission and average delay for variable-length transmission.
  • The architecture is fully convolutional, supports layered transmission, and remains robust to variations in feedback-channel quality.

II. Problem Formulation

The paper formulates wireless image transmission with noisy or noiseless channel-output feedback, seeking codes that minimize average image distortion under fixed system parameters. It also defines fixed-length and feedback-enabled variable-length transmission settings.

  • An input image x ∈ R^n is transmitted in k uses of a noisy complex channel and reconstructed at the receiver.The image dimension n is the source bandwidth, while k is the channel bandwidth and k/n is the bandwidth ratio.
  • The transmitter receives a noisy version of the channel output with unit delay and uses previous feedback values when generating each channel input.The feedback channel is modeled as w = z + n_f.
  • The decoding function maps the received channel output to a reconstructed image, with performance measured by a specified distortion metric between input and output images.
  • The study focuses on bandwidth compression, k < n, while allowing static and fading memoryless complex AWGN channel models.For fading channels, the channel gain remains constant throughout transmission of a single image and is random across transmissions.
  • The formulation supports fixed-length coding and variable-length coding that minimizes average code length while meeting a target distortion for each image.The variable-length setting is enabled by feedback and is considered later in the paper.

III. DeepJSCC-f Architecture

DeepJSCC-f is a layered convolutional autoencoder that uses feedback-derived receiver estimates to refine image reconstruction across transmissions. Intermediate estimates support sequential training and variable-length stopping once a target quality is reached.

  • DeepJSCC-f is a DNN-based JSCC scheme that exploits channel-output feedback through layered autoencoders.The paper identifies this layered use of feedback as a key architectural innovation.
  • The architecture transmits L partial layers, with each layer intended to improve the receiver's estimate; increasing L provides more feedback but makes each layer shorter and harder to optimize.When L = 1, feedback is unused; when L = 2, feedback affects only the second half of the channel block.
  • For layers j > 1, the encoder receives the original image and the transmitter's estimate of the receiver reconstruction from the previous layer.The transmitter reconstructs this estimate using the previous decoder and combiner applied to feedback, then concatenates it with the source image along the channel axis.
  • Each layer contains an encoder and decoder, while layers j ≥ 2 additionally use a combiner to merge current and previous reconstruction outputs.The receiver's decoder uses all channel outputs received so far.
  • Intermediate estimates enable sequential layer training and variable-length coding that stops transmission when the receiver reaches the target reconstruction quality.
  • The encoder can learn to transmit either image content or error-correction and refinement information by receiving both the source image and the current estimate.The authors report that providing both inputs rather than imposing residual-only coding significantly improves performance.

IV. Experimental Results

The experiments evaluate DeepJSCC-f across feedback conditions, compression rates, channel models, and SNR values using CIFAR10 images and comparisons against feedback-agnostic schemes or theoretical bounds. The authors present it as the first practical JSCC implementation able to exploit channel-output feedback.

  • The evaluation covers noiseless and noisy feedback, different compression rates, AWGN and fading channels, and multiple channel SNR values.
  • DeepJSCC-f is reported as the first practical JSCC implementation in the literature able to transmit images while exploiting channel-output feedback.Because the authors state there are no direct competitors, comparisons use schemes that ignore feedback or theoretical capacity-achieving bounds.
  • Plotted results average 10 channel realizations for every evaluation image, and image quality is measured using PSNR.For the 24-bit RGB images, the maximum pixel value used in PSNR is 255.
  • Models are trained with Adam and evaluated after validation performance stops decreasing, using mean squared error as the layer training loss.The implementation uses a learning rate of 10^-4 and batch size 128.

A. DeepJSCC-f with Two Layers (L = 2)

For the two-layer setting with one use of perfect feedback, DeepJSCC-f first transmits a base layer and then sends a second message conditioned on the first layer's channel output. The experiment fixes equal layer lengths and evaluates compression rate k/n = 1/6.

  • With perfect feedback used once, the image is transmitted in two layers: a base layer y1 followed by a second message y2 conditioned on the first layer's output.The feedback for the second transmission is w1 = z1.
  • The two layers are assigned equal lengths, k1 = k2.
  • The reported two-layer experiment uses compression rate k/n = 1/6.

1) General Performance:

DeepJSCC-f uses feedback within a joint source-channel architecture and improves image reconstruction over single-transmission and separation-based schemes, especially at low SNR. Its gains persist against strong practical and theoretical separation baselines.

  • Architecture: DeepJSCC-f uses feedback-enhanced joint source-channel transmission, with separate encoders, decoders, and combiners trained for each SNR.The architecture is evaluated against schemes combining established image codecs with LDPC+QAM or capacity-achieving channel codes.
  • Feedback gains: At k/n = 1/6, DeepJSCC-f improves over the corresponding scheme without feedback by at least 1dB, particularly in the low-SNR regime.The compared single-transmission architecture is the L = 1 special case using a capacity-achieving channel code.
  • Robustness: DeepJSCC-f surpasses BPG with a capacity-achieving channel code, particularly at low SNR, and the improvement is attributed to JSCC.The comparison is favorable to separation because the capacity-achieving code is hypothetical and assumes infinite blocklength, whereas the experiments use short blocklengths.

2) Graceful Degradation:

DeepJSCC-f behaves more like analog transmission than separation-based digital communication: reconstruction quality degrades gradually as deployment SNR worsens, rather than collapsing at a threshold.

  • Analog-like behavior: DeepJSCC-f exhibits graceful degradation with channel SNR, unlike the threshold behavior of digital separation-based schemes.This analog-like behavior is presented as an advantage over digital schemes.
  • SNR mismatch: Models trained at SNRtrain = 1dB and SNRtrain = 7dB improve when evaluated at higher SNRtest values.The evaluation spans SNRtest values from -2dB to 13dB.
  • SNR mismatch: As SNRtest decreases below the training condition, DeepJSCC-f degrades without a rapid collapse and still produces recognisable images at very low SNRs.The corresponding digital scheme deteriorates drastically beyond a channel-quality threshold because the receiver cannot decode the channel and source codes.

3) Fading Channel:

DeepJSCC-f remains effective over slow Rayleigh fading channels and can use feedback to improve performance and channel estimation without explicitly transmitting pilots.

  • Channel model: The fading-channel evaluation uses a slow Rayleigh channel whose gain remains constant during transmission of one image.The comparison considers different average channel SNRs and a bandwidth ratio of k/n = 1/6.
  • Channel estimation: DeepJSCC-f learns an efficient fading-channel transmission scheme without explicit pilots or channel estimation, whereas separation assumes perfect receiver CSI.The cited comparison concerns prior single-layer DeepJSCC results and the separation-based assumption.
  • Feedback: More feedback transmissions indirectly improve channel estimation and performance in fading channels, even though pilots are not explicitly transmitted.The figure studies different numbers of layers at k/n = 0.5.
  • Layer depth: Adding layers initially increases performance, but gains stabilize and can decline beyond a threshold.Increasing L also increases complexity because separate encoder, decoder, and combiner networks must be trained for each layer.
  • Layer depth: L = 4 layers typically provides a reasonable performance trade-off between reconstruction quality and model complexity.The experiments evaluate performance over channel SNR values using k/n = 1/3.
  • Image-level results: DeepJSCC-f improves reconstruction quality for the vast majority of transmitted CIFAR-10 images and outperforms other benchmarks across tested settings.Positive performance gaps denote images for which DeepJSCC-f achieves higher PSNR than its competitor.

2) Variable Rate Transmission:

DeepJSCC-f supports variable-rate transmission by refining reconstructions layer by layer and stopping when a target distortion is met. It reduces the bandwidth needed to reach target PSNR compared with separation-based transmission.

  • Variable-rate mechanism: After each layer, the receiver deterministically reconstructs the image, allowing transmission to stop when the distortion target is met.The encoder can determine whether further information is needed after each layer.
  • Evaluation: DeepJSCC-f is compared with headerless BPG followed by an ideal capacity-achieving code when measuring bandwidth required for target PSNR.The digital baseline compresses each image to the minimum bits meeting the target and computes the channel uses needed to transmit them.
  • Bandwidth efficiency: Almost all images require less channel bandwidth with DeepJSCC-f than with separation, despite transmission stopping only after a complete layer.The experiment evaluates target PSNR values of 20dB, 25dB, and 30dB.
  • Trade-off: Reducing the number of channel uses per layer would provide finer bandwidth granularity, at the cost of increased complexity.The current layered scheme has low granularity because transmission can stop only after a whole layer.
  • Bandwidth efficiency: For a 30 dB PSNR target, DeepJSCC-f requires an average bandwidth ratio almost half that required by the separation-based bound.The gap between the curves increases with the PSNR target.
  • Short-blocklength regime: At PSNR 20 dB and channel SNR = 10 dB, almost all image bits must be transmitted within 307 channel uses under the separation bound with k/n = 0.1.This example illustrates why the separation bound can be loose at short blocklengths.

3) Fading Channel:

DeepJSCC-f improves transmission over fading channels as the number of feedback layers increases, while retaining robustness to noisy and mismatched feedback conditions. Feedback can provide channel-state information in fading settings, and performance degrades gracefully as feedback quality declines.

  • Fading-channel performance: Performance over the fading channel improves significantly compared with the L = 2-layer results on the AWGN channel.
  • Fading-channel performance: Increasing the number of layers beyond L = 4 benefits fading-channel performance, unlike the observed AWGN-channel behavior.The authors attribute this to feedback conveying both channel corruption information and channel-condition information usable for channel estimation.
  • Noisy feedback: At SNRfb = 20 dB, performance is only slightly below noiseless-feedback transmission; as feedback SNR decreases, performance degrades but remains competitive.
  • Noisy feedback: At SNRfb = 0 dB, additional layers still refine reconstruction, but overall performance falls below the single-transmission case with L = 1.A hybrid encoder that selectively uses feedback was proposed, but its implementation was left for future work.
  • Feedback-channel mismatch: Models remain useful under substantial training–test feedback-SNR mismatch, and a model trained without feedback noise can surpass the single-transmission bound at SNRfb = 10dB in low forward SNRs.
  • Feedback-channel mismatch: DeepJSCC-f exhibits graceful degradation with feedback-channel quality, although sending an extra layer can reduce performance in extremely adverse mismatched conditions with high forward SNRs.Even then, the retrieved signal can remain identifiable, with failed transmission PSNR approximately 15dB.

D. Model Generalisation to Bigger Datasets

The fully convolutional DeepJSCC-f architecture generalizes beyond small image datasets: it is trained on ImageNet and evaluated on Kodak images while progressively refining reconstructions across feedback stages. The authors report higher reconstruction quality than separation-based schemes and broader adaptability across channels and datasets.

  • Dataset generalization: DeepJSCC-f accepts multiple input dimensions because its architecture is fully convolutional, rather than being limited to a specific input size or type.
  • Dataset generalization: The model was trained on ImageNet, a dataset described as containing 1.2 milliom images across over 20000 classes.Training used 128 × 128 patches in batches of 16 images.
  • Comparison with separation: DeepJSCC-f produces higher-quality Kodak reconstructions with increased detail and better PSNR, SSIM, and MS-SSIM performance than the compared separation-based schemes.The visual gains are reported particularly for high-frequency image elements.
  • Broader capabilities: The work reports flexible strategies including variable-rate transmission, analog behavior, graceful degradation, and adaptation to forward or feedback channel variations.The scheme is also reported to adapt to fading channels and diverse datasets.
  • Progressive reconstruction: With ImageNet-trained models evaluated on Kodak images over an AWGN channel at SNR = 1 dB, later transmission stages produce progressively enhanced reconstructions.The second decoder receives a residual-like refinement, which the combiner merges with the first reconstruction to produce x̂2.
  • Future directions: The authors characterize this work as a first step, with further improvements expected from advances in model architecture and training strategy.

Appendix A

The appendix compares DeepJSCC-f with practical separation-based digital systems and visualizes reconstructed-image quality at fixed AWGN-channel settings. DeepJSCC-f is reported to outperform the compared schemes across the considered image-quality metrics, especially in high-frequency details.

  • Evaluation procedure: For each channel-code configuration, the evaluation computes a maximum image rate Rmax, measures frame-error probability ϵ, and averages reconstruction performance over the dataset.
  • Visual comparison: The visual comparison uses AWGN SNR=1dB, bandwidth compression k/n = 1/6, noiseless feedback for DeepJSCC-f, and a 1/3-rate LDPC + 4QAM separation baseline.
  • Results: DeepJSCC-f presents superior performance in all considered metrics, including PSNR, SSIM, and MS-SSIM, for the compared images.
  • Results: The reported visual advantage is particularly apparent in high-frequency elements such as bushes, leaves, water, background landscapes, and house details.
  • Metric optimization: The presented DeepJSCC-f results were optimized for MSE, while direct optimization for SSIM or MS-SSIM is described as capable of producing superior results for those metrics.
Loading 1911.11174v2…