PaperlayerView exact revision on arXiv

Source-linked AI summary

CDDM: Channel Denoising Diffusion Models for Wireless Semantic Communications

Tong Wu, Zhiyong Chen, Dazhi He, Liang Qian, Yin Xu, Meixia Tao, Wenjun Zhang

arXiv:2309.08895v1cs.ITeess.SP

TL;DR

The paper asks whether diffusion models can mitigate wireless channel noise, addressing limited prior use of diffusion models in wireless communications. It proposes channel-adapted CDDM after equalization, derives training and sampling procedures, and applies CDDM to JSCC image transmission, where experiments report lower MSE and better overall performance than the compared systems.

  • Problem

    Prior diffusion-model research has focused mainly on AIGC, while wireless communication systems still need effective channel-noise mitigation beyond direct JSCC optimization.

  • Method

    CDDM learns the conditional distribution of received signals using channel-adapted forward diffusion and reverse sampling, then denoises equalized signals before JSCC decoding.

  • Results

    The joint CDDM and JSCC system achieves lower MSE than systems without CDDM under AWGN and Rayleigh fading, with improved performance over JSCC and JPEG2000+LDPC comparisons.

  • Takeaways & Limitations

    CDDM provides a channel-specific physical-layer denoising module that improves the evaluated JSCC-based semantic image transmission system.

Abstract

from arXiv · show

Diffusion models (DM) can gradually learn to remove noise, which have been widely used in artificial intelligence generated content (AIGC) in recent years. The property of DM for eliminating noise leads us to wonder whether DM can be applied to wireless communications to help the receiver mitigate the channel noise. To address this, we propose channel denoising diffusion models (CDDM) for semantic communications over wireless channels in this paper. CDDM can be applied as a new physical layer module after the channel equalization to learn the distribution of the channel input signal, and then utilizes this learned knowledge to remove the channel noise. We derive corresponding training and sampling algorithms of CDDM according to the forward diffusion process specially designed to adapt the channel models and theoretically prove that the well-trained CDDM can effectively reduce the conditional entropy of the received signal under small sampling steps. Moreover, we apply CDDM to a semantic communications system based on joint source-channel coding (JSCC) for image transmission. Extensive experimental results demonstrate that CDDM can further reduce the mean square error (MSE) after minimum mean square error (MMSE) equalizer, and the joint CDDM and JSCC system achieves better performance than the JSCC system and the traditional JPEG2000 with low-density parity-check (LDPC) code approach.

I. INTRODUCTION

The paper proposes CDDM to mitigate wireless channel noise by adapting diffusion processes to conditional received-signal distributions and integrating denoising with JSCC-based semantic communications. It develops the method theoretically and evaluates its effects on signal MSE and image transmission.

  • Proposed approach: CDDM starts reverse sampling from the received signal, reducing the number of reverse sampling steps compared with starting from pure noise.This design leverages the received signal while preserving the denoising role of the reverse process.
  • Motivation: Few prior works apply diffusion models to wireless communications, while semantic communications and deep-learning JSCC address efficient transmission of semantic information.The paper positions CDDM at the intersection of diffusion-based modeling, wireless communication, and semantic image transmission.
  • Proposed approach: CDDM is placed after channel equalization to learn the channel-input distribution and remove noise from received signals.The method uses an explicit conditional probability model of the received signal and can operate under AWGN and Rayleigh fading conditions.
  • Proposed approach: The proposed forward diffusion and reverse sampling processes are designed from channel-specific conditional distributions rather than directly reused from conventional diffusion models.The training objective uses a variational lower bound, while Bayesian conditional posteriors yield the sampling algorithm.
  • Analysis and evaluation: The paper derives a sufficient condition for conditional-entropy reduction and applies CDDM with JSCC for wireless image transmission.Experiments assess MSE after denoising and compare the joint system with JSCC and JPEG2000+LDPC approaches.

II. SYSTEM MODEL

The system model represents transmitted complex symbols as real vectors, transmits them over Rayleigh or AWGN channels, and applies MMSE equalization followed by normalization and reshaping. The resulting conditional received-signal distributions provide the basis for CDDM design.

  • Signal representation: The transmitted real vector x ∈ R2k represents k complex symbols through paired real and imaginary components.The i-th complex symbol is formed as xc,i = xi + jxi+k.
  • Wireless channel: The complex channel model uses Rayleigh fading gains and additive Gaussian noise, with transmitted symbols subject to a power constraint.The fading gains are i.i.d. and the noise samples are i.i.d. complex Gaussian.
  • Receiver processing: The receiver equalizes yc with MMSE and then applies normalization and reshaping to produce the real vector yr.The receiver is assumed to obtain the channel state through channel estimation.
  • Conditional model: The conditional distribution of yr given x and channel state is derived for Rayleigh fading and AWGN channels to guide CDDM’s diffusion processes.These channel-specific distributions determine how the forward diffusion and reverse sampling procedures are constructed.
  • CDDM interface: The trained CDDM estimates the standardized noise and uses the noise coefficient matrix in a sampling algorithm intended to recover the signal component.The estimated noise is produced by ϵθ(·), whose parameters are learned during CDDM training.

III. CHANNEL DENOISING DIFFUSION MODELS

This section presents the proposed CDDM forward and reverse processes, including their training and sampling algorithms and the condition under which reverse sampling reduces received-signal conditional entropy.

  • Overview: The CDDM forward diffusion and reverse sampling processes are described before deriving a sufficient condition for conditional-entropy reduction.The section treats the diffusion procedure as a channel-adapted denoising mechanism.

A. Training Algorithm of CDDM

The CDDM training process constructs a channel-matched forward diffusion process and trains a noise-prediction model by optimizing a variational likelihood bound.

  • A. Training Algorithm of CDDM: The forward process is designed so the diffused signal follows the same distribution as the real channel-and-equalizer output.This lets CDDM train on x_m rather than directly on the received signal y_r, while decomposing sampling into m steps.
  • A. Training Algorithm of CDDM: The training algorithm samples x, a diffusion step t, channel-related quantities, and Gaussian noise before updating the model parameters.The update uses the noise-prediction loss defined by the proposed objective.
  • A. Training Algorithm of CDDM: The training objective maximizes a variational lower bound on the log-likelihood using latent variables introduced by the forward diffusion process.The bound is decomposed into terms including reconstruction and KL-divergence components.
  • A. Training Algorithm of CDDM: CDDM is trained to estimate channel noise through the parameterized reverse distribution pθ(x_t−1|x_t,h_r).The learned posterior approximation is converted into noise estimation, producing the model output ϵθ(x_t,h_r,t).

B. Sampling Algorithm of CDDM

The CDDM sampling procedure initializes the reverse process with the received signal and repeatedly predicts noise to recover the transmitted signal.

  • B. Sampling Algorithm of CDDM: The sampling process sets x_m=y_r and performs reverse updates for t=m,...,2 using the trained noise predictor.The predicted noise is used to sample x_t−1 at each step.
  • B. Sampling Algorithm of CDDM: Because x_0 is unavailable during sampling, CDDM estimates it by replacing the true noise with ϵθ(x_t,h_r,t).This estimated x_0 is then used in the reverse sampling expression.
  • B. Sampling Algorithm of CDDM: At the final step t=1, the algorithm predicts x_0 directly rather than continuing the general reverse update.The resulting x_0 is returned as the recovered signal.
  • B. Sampling Algorithm of CDDM: The maximum sampling step is selected so the model remains within the sufficient-condition range under the AWGN channel.Figure 4 marks this maximum step with a black dot.

C. Analysis on the conditional entropy

The conditional-entropy analysis relates each diffusion step to received-signal uncertainty and shows that denoising benefits require suitable sampling-step choices and stated assumptions.

  • C. Analysis on the conditional entropy: The analysis compares H(x_t,i|x_0,h) with an upper bound on H(x_t−1,i|x_0,h) to characterize entropy changes during denoising.The received signal is identified with x_t because the forward process can match the received-signal conditional distribution.
  • C. Analysis on the conditional entropy: The theoretical result assumes a bounded element-wise noise-prediction loss and zero mathematical expectation for the network output.These assumptions support the sufficient-condition analysis.
  • C. Analysis on the conditional entropy: Theorem 1 establishes a sufficient condition under which the conditional entropy decreases across sampling steps.The condition is derived from the cross-correlation and variance relationships in the proof.
  • C. Analysis on the conditional entropy: The recommended maximum sampling step is t_max∈[10,150], because excessive steps can exceed the threshold, while too few steps may leave channel noise insufficiently removed.The analysis also notes that a flattened entropy curve can produce no performance improvement.

IV. THE JOINT CDDM AND JSCC FOR SEMANTIC COMMUNICATIONS

The paper applies CDDM within a JSCC-based semantic communications system for wireless image transmission.

  • IV. THE JOINT CDDM AND JSCC FOR SEMANTIC COMMUNICATIONS: The proposed CDDM is integrated into a JSCC semantic communications system for wireless image transmission.This application evaluates CDDM as part of an end-to-end image transmission system.

A. System Structure

The joint system encodes an RGB image into a transmitted signal with a JSCC encoder, processes it through the channel, and feeds the received signal to the receiver pipeline.

  • A. System Structure: The JSCC encoder maps an RGB source image s to transmitted signal x ∈R2k.The encoder uses a Swin Transformer backbone with shifted-window multi-head self-attention.
  • A. System Structure: The transmitted signal is processed through the wireless channel to produce the receiver-side signal yr.
  • A. System Structure: The system architecture combines JSCC-based image transmission with receiver-side processing of the channel output.

B. Training algorithm

Training proceeds in three stages: JSCC pretraining, CDDM training with a fixed encoder, and joint decoder retraining with the trained CDDM.

  • B. Training algorithm: Algorithm 3 uses training-set samples, diffusion hyperparameters, channel estimation results, and noise power as inputs.The algorithm outputs the well-trained joint CDDM and JSCC system.
  • B. Training algorithm: The first stage jointly trains the JSCC encoder and decoder through the channel without CDDM to minimize d(s,ŝ).The loss parameters ϕ and φ represent the JSCC encoder and decoder parameters.
  • B. Training algorithm: The second stage fixes the JSCC encoder and trains CDDM to learn the distribution of x0 using Algorithm 1.Its special forward diffusion process simulates channel-noise distributions, allowing one training process across channel conditions.
  • B. Training algorithm: The third stage retrains the JSCC decoder jointly with the trained encoder and CDDM through the real channel.Only decoder parameters are updated while minimizing the reconstruction distance d(s,ŝ).

C. Model Structure

The system combines a hierarchical Swin-Transformer JSCC architecture with a convolutional improved U-Net for CDDM processing.

  • C. Model Structure: The JSCC encoder partitions images into non-overlapping patches and extracts semantic features through M Basicblocks containing patch merging and Swin Transformers.Each Basicblock halves feature height and width while increasing channel dimensions.
  • C. Model Structure: The JSCC decoder mirrors the encoder, replacing encoder downsampling modules with upsampling modules.
  • C. Model Structure: CDDM uses a convolutional improved U-Net that processes yr through convolutional layers before producing the final output y.Its components include Conv-Res blocks, Conv-Attn blocks, downsampling blocks, and upsampling blocks.
  • C. Model Structure: The experiments evaluate the proposed system across datasets and channel conditions, including channel estimation errors and different SNRs.

A. Experiment Setup

Experiments compare joint CDDM and JSCC with JSCC alone and JPEG2000+LDPC across image datasets, channel conditions, SNRs, and channel-estimation errors. The results show lower MSE and improved PSNR, MSSSIM, color consistency, and visual quality for the joint system, with gains decreasing under larger estimation errors.

  • A. Experiment Setup: The evaluation uses CIFAR10 and DIV2K, covering low- and high-resolution image transmission.CIFAR10 contains 50,000 training and 10,000 testing images, while DIV2K contains 800 training and 100 testing images.
  • A. Experiment Setup: The comparison includes joint CDDM and JSCC, JSCC without CDDM, and JPEG2000+LDPC.The two JSCC systems share the same structure and training configuration, while the classical baseline uses DVB-T2 LDPC codes.
  • A. Experiment Setup: Performance is measured using PSNR and MSSSIM, with higher values indicating better reconstructed-image quality or similarity.MSSSIM is converted to dB for testing and comparison.
  • B. MSE performance and visualization results: Under Rayleigh fading at SNR=10 dB, both JSCC-based systems have better visual quality than JPEG2000+LDPC, while joint CDDM and JSCC improves color consistency.The comparison uses reconstructed images at slightly lower CBR than the classical system.
  • B. MSE performance and visualization results: 0.27 dB: CDDM reduces MSE by 0.27 dB at SNR=20 dB under AWGN with CBR 3/128.The system with CDDM performs better than the system without CDDM across SNR regimes under both AWGN and Rayleigh fading.

C. PSNR performance

The joint CDDM and JSCC system improves image-transmission quality over JSCC across channel conditions, SNRs, CBRs, datasets, and evaluation metrics, while gains weaken under high estimation error or larger CBRs.

  • PSNR under AWGN: 0.5 dB at SNR=20 dB: joint CDDM and JSCC exceeds JPEG2000+LDPC, while JPEG2000+LDPC is comparable to JSCC.Under AWGN for DIV2K, joint CDDM and JSCC also outperforms JSCC from 5 to 20 dB.
  • PSNR under Rayleigh fading: 0.83 dB for CIFAR10 and 0.53 dB for DIV2K at SNR=10 dB: joint CDDM and JSCC outperforms JSCC under Rayleigh fading with perfect channel estimation.The comparison covers both datasets across all SNRs.
  • PSNR across CBRs: The system remains effective across CBRs, but its performance gain decreases as CBR increases because the fixed CDDM architecture struggles with more complex transmitted-signal distributions.This result is reported for high-resolution DIV2K under AWGN and Rayleigh fading at 10 dB.
  • PSNR across SNRs: Joint CDDM and JSCC consistently outperforms JSCC across varying SNRs, with larger gains at lower SNRs under Rayleigh fading.The CDDM adjusts its sampling step when SNR varies, supporting adaptability across SNR conditions.
  • MSSSIM performance: Under MSSSIM, joint CDDM and JSCC outperforms JSCC and JPEG2000+LDPC across the tested conditions, including AWGN, Rayleigh fading, SNR, and CBR settings.The reported MSSSIM gains include 0.6 dB at SNR=15 dB under AWGN for DIV2K.
  • Overall experimental findings: Across SNRs, CBRs, metrics, channel types, and image resolutions, joint CDDM and JSCC surpasses both JSCC and JPEG2000+LDPC.Experiments were conducted on CIFAR10 and DIV2K using MSE, PSNR, and MSSSIM.
Loading 2309.08895v1…