PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Joint Source-Channel Coding for Wireless Image Transmission

Eirina Bourtsoulatze, David Burth Kurka, Deniz Gunduz

arXiv:1809.01733v4cs.ITcs.LGeess.SPstat.ML

TL;DR

Wireless image transmission must operate under severe latency, bandwidth, and energy constraints that challenge separate compression and error-correction pipelines. The paper proposes deep JSCC, which jointly learns convolutional encoding and decoding while modeling the noisy channel as a non-trainable layer. It reports superior performance to separation-based digital transmission in low-SNR and limited-bandwidth AWGN settings, graceful degradation across SNR changes, and strong gains over slow Rayleigh fading channels.

  • Problem

    Emerging wireless applications require image and video transmission under extreme latency, bandwidth, and energy constraints that can preclude computationally demanding long-blocklength source and channel coding.

  • Method

    Deep JSCC directly maps image pixels to complex-valued channel inputs using jointly trained convolutional neural networks with the noisy channel represented as a non-trainable layer.

  • Results

    Deep JSCC outperforms separation-based digital transmission especially at low SNR and limited bandwidth, avoids the cliff effect, and significantly outperforms separation-based transmission over slow Rayleigh fading.

  • Takeaways & Limitations

    The learned joint representation remains robust as channel quality varies, including settings where digital separation breaks down or cannot exploit improved channel conditions.

Abstract

from arXiv · show

We propose a joint source and channel coding (JSCC) technique for wireless image transmission that does not rely on explicit codes for either compression or error correction; instead, it directly maps the image pixel values to the complex-valued channel input symbols. We parameterize the encoder and decoder functions by two convolutional neural networks (CNNs), which are trained jointly, and can be considered as an autoencoder with a non-trainable layer in the middle that represents the noisy communication channel. Our results show that the proposed deep JSCC scheme outperforms digital transmission concatenating JPEG or JPEG2000 compression with a capacity achieving channel code at low signal-to-noise ratio (SNR) and channel bandwidth values in the presence of additive white Gaussian noise (AWGN). More strikingly, deep JSCC does not suffer from the ``cliff effect'', and it provides a graceful performance degradation as the channel SNR varies with respect to the SNR value assumed during training. In the case of a slow Rayleigh fading channel, deep JSCC learns noise resilient coded representations and significantly outperforms separation-based digital communication at all SNR and channel bandwidth values.

I. Introduction

Deep JSCC addresses latency and bandwidth constraints by jointly learning image transmission over wireless channels, replacing separate compression and channel coding with direct neural mapping. Experiments report advantages over separation-based digital transmission, especially under low SNR, limited bandwidth, and varying channel quality.

  • Motivation: Conventional systems separately compress image data, apply error correction, and modulate the resulting bitstream.This modular pipeline is theoretically optimal for long source and channel blocks but can be computationally demanding in constrained applications.
  • Proposed approach: Deep JSCC directly maps image pixel values to complex-valued channel inputs using jointly trained convolutional encoder and decoder networks.The noisy channel is incorporated as a non-trainable neural-network layer, with AWGN and slow Rayleigh fading considered.
  • Results: Deep JSCC outperforms JPEG or JPEG2000 combined with capacity-achieving channel codes at low SNR and limited bandwidth over time-invariant AWGN channels.The comparison assumes the separation scheme operates at channel capacity despite short blocklengths.
  • Results: Deep JSCC provides graceful degradation as channel SNR varies and avoids the cliff effect observed in digital transmission.This behavior is relevant when receivers experience different channel qualities or when channel conditions are unknown.
  • Scope and significance: The architecture is presented as an end-to-end system for high-resolution wireless image transmission over AWGN and fading channels.The authors also describe extensions to other performance measures and source signals, including video.

II. Background and Problem Formulation

The paper formulates point-to-point image transmission as mapping an n-dimensional image to k complex channel symbols and reconstructing it after channel corruption. It contrasts this learned joint mapping with separate digital pipelines whose quality can degrade sharply when channel conditions differ from design assumptions.

  • System model: The transmitter maps an image x ∈ R^n to k complex-valued channel symbols z, with k/n defining the bandwidth compression ratio.Typically k < n, and the transmitted signal may satisfy an average or peak power constraint.
  • Conventional pipeline: Conventional transmission independently performs source encoding, channel coding, and modulation before reversing those operations at the receiver.Typical components include JPEG or JPEG2000, LDPC or Turbo codes, and matched modulation and demodulation.
  • Separate-system limitation: Separate coding targets a specific channel quality by choosing source and channel rates jointly around the assumed channel capacity.When the experienced channel is worse than assumed, the channel-code error probability increases rapidly and reconstruction quality drops.
  • Separate-system limitation: Once separate coding rates are fixed, improved channel conditions do not improve reconstruction quality above the target rate.This behavior, together with abrupt deterioration below the target condition, constitutes the cliff effect.
  • Proposed formulation: The proposed approach bypasses pixel-to-bit conversion by directly mapping real pixel values to complex-valued channel inputs.This uncoded-transmission-inspired design seeks a joint source-channel representation rather than separate bitstream processing.

III. DL-based JSCC

Deep JSCC uses jointly trained CNN encoder and decoder functions with the noisy communication channel embedded as a non-trainable neural-network layer. The system directly transmits learned complex-valued representations and optimizes reconstruction distortion end to end.

  • Deep JSCC directly maps image pixels to complex-valued channel symbols through a CNN-based encoder and decoder.
  • The encoder transmits the real and imaginary parts of each complex symbol over the signal’s I and Q components.
  • The channel is modeled as non-trainable layers within the neural-network architecture, enabling end-to-end optimization through communication impairments.
  • The decoder maps the corrupted complex-valued signal back to an estimate of the original image using transpose convolutional layers.
  • The encoder and decoder are jointly designed to minimize expected distortion between original and reconstructed images, estimated by sampling an available dataset.

IV. Evaluation

The evaluation implements deep JSCC with convolutional and transpose-convolutional architectures, varying the final encoder filters to obtain different bandwidth compression ratios. Performance is measured by PSNR under normalized signal-power conditions.

  • The implementation uses the encoder and decoder neural-network architectures shown in Fig. 2.
  • Input images are normalized to the [0, 1] range, and the encoder’s final layer uses 2k units for complex channel inputs.
  • The decoder combines real and imaginary channel outputs before transpose convolutions progressively reconstruct the image at its original resolution.
  • Different bandwidth compression ratios k/n are obtained by varying the number of filters in the encoder’s last convolutional layer.
  • Performance is quantified using PSNR, while experiments use 24-bit RGB images with maximum pixel value MAX = 255.
  • The average signal power is set to P = 1 for all experiments, including the proposed and benchmark schemes.

A. Evaluation on CIFAR-10 dataset

On CIFAR-10, deep JSCC is evaluated against digital JPEG/JPEG2000 transmission over AWGN and slow Rayleigh fading, including robustness to channel-SNR mismatch. It remains effective in challenging bandwidth and channel conditions, while avoiding abrupt degradation as AWGN SNR worsens.

  • Experimental setup: 50,000 training and 10,000 distinct test images are used from CIFAR-10 to evaluate the proposed scheme.Training combines images with random channel realizations.
  • AWGN evaluation: The AWGN comparison uses an upper bound for digital schemes employing JPEG or JPEG2000, based on Shannon separation and channel capacity.The bound assumes reliable transmission at the maximum rate and accounts for JPEG/JPEG2000 bitrate limits.
  • AWGN evaluation: At SNR=0dB, JPEG/JPEG2000 digital schemes break down, whereas deep JSCC remains capable of transmission with reasonably good performance.At medium and high SNR with k/n ∈[0.04, 0.2], deep JSCC is considerably above the digital upper bound; for k/n > 0.3, it remains competitive.
  • Robustness to SNR mismatch: For SNRtest < SNRtrain, deep JSCC avoids the digital cliff effect and degrades gradually as AWGN channel quality worsens.The paper attributes this behavior to mapping similar images or features to nearby channel-input points.
  • Robustness to SNR mismatch: For SNRtest > SNRtrain, reconstruction quality first improves gradually and then saturates at an SNR above the training value.The saturation level is determined by the compression implicitly learned for SNRtrain and the bottleneck ratio k/n.
  • Slow Rayleigh fading: Deep JSCC beats the digital upper bound at all SNR and bandwidth-compression values on the slow Rayleigh fading channel without channel state information.The experiment uses random channel gains and no pilot signals; the paper notes benefits for time-varying channels and multicasting.

B. Evaluation on the Kodak dataset

On the Kodak dataset, deep JSCC is evaluated against JPEG/JPEG2000-based digital transmission over AWGN and slow Rayleigh fading channels. It performs especially well at low SNR and limited bandwidth, avoids the digital cliff effect, and produces competitive visual quality.

  • Setup: Deep JSCC is evaluated on 24 Kodak images of resolution 768 × 512, with each image transmitted 100 times for averaging.The network is trained on 128×128 ImageNet patches before Kodak evaluation.
  • AWGN results: At k/n = 1/12 and 1/6, deep JSCC significantly outperforms JPEG-based digital schemes at low SNR and low channel bandwidth, while matching them at high values.The baselines combine JPEG or JPEG2000 compression with channel coding and modulation.
  • AWGN results: Deep JSCC avoids the digital cliff effect, degrading gracefully below the training SNR instead of suffering abrupt reconstruction-quality loss.Its performance also does not saturate immediately when channel conditions improve beyond the target SNR.
  • AWGN results: Against JPEG2000 baselines, deep JSCC outperforms only in very low SNR regimes and at low channel bandwidth over AWGN.The authors suggest deeper networks and more sophisticated activation and loss functions as potential improvements.
  • Visual quality: In visual comparisons, deep JSCC avoids JPEG blocking artifacts at low SNR, while its smoother noise can yield comparable SSIM despite lower high-SNR PSNR.Deep JSCC can also be trained with SSIM as the loss function to improve SSIM performance.

C. Computational complexity

The deep JSCC encoder and decoder have computational complexity linear in input image size. Its measured runtime is competitive with JPEG and lower than reported JPEG2000 runtimes, excluding additional channel-code processing.

  • Complexity: The computational cost of the convolutional encoder/decoder is linear in the number of input pixels.Only feature-map width and height depend on image dimensions; other factors remain constant.
  • Complexity: The separate JPEG-plus-LDPC scheme is also linear in input-image size, so both approaches share the same asymptotic scaling.The comparison therefore concerns measured runtime and additional coding stages rather than asymptotic order alone.
  • Runtime: 18ms per image on GPU and 387ms on CPU are reported for deep JSCC on 768 × 512 Kodak images.The measurements use a Tesla K80 GPU and eight 2.10GHz Intel Xeon E5-2620V4 CPUs.
  • Runtime: JPEG encoding and decoding reportedly takes 30–390ms, while JPEG2000 takes 430–590ms, before channel-code time is added.These figures are reported for the same Kodak image resolution.

V. Conclusions and Future Work

The paper concludes that deep JSCC directly learns image transmission and provides strong performance at low SNR, limited bandwidth, and slow fading. Future work targets stronger architectures and broader channel models.

  • V. Conclusions and Future Work: Deep JSCC directly maps images to channel inputs using jointly trained complementary CNN encoder and decoder networks.Training minimizes average reconstructed-image MSE.
  • V. Conclusions and Future Work: Deep JSCC outperforms separation-based schemes especially at limited bandwidth and low SNR, while degrading gracefully with channel SNR.Over slow fading, it outperforms the separation-based scheme at any channel bandwidth value.
  • V. Conclusions and Future Work: With channel noise, the encoder-decoder learns non-trivial representations that are resilient to noise, including when the channel is wider than the source.This behavior is analogous to denoising autoencoders.
  • Future work: The next improvement target is using more advanced neural-network architectures that have demonstrated better compression performance.The authors also plan to study non-Gaussian channels and channels with memory.
  • Future work: The current scope excludes non-Gaussian channels and channels with memory lacking capacity-approaching channel codes.The authors expect the benefits of neural JSCC to be more evident in these non-ideal settings.

Original Deep JSCC JPEG JPEG2000

Figure 11 presents reconstructed images from deep JSCC, JPEG, and JPEG2000 at two channel bandwidth settings across increasing SNR.

  • Original Deep JSCC JPEG JPEG2000: Figure 11 compares reconstructed images for k/n = 1/12, with rows ordered from 1dB to 19dB SNR.The compared methods are deep JSCC and digital schemes using JPEG/JPEG2000 compression.
Loading 1809.01733v4…