An Iterative BP-CNN Architecture for Channel Decoding

Fei Liang; Cong Shen; Feng Wu

Source-linked AI summary

An Iterative BP-CNN Architecture for Channel Decoding

Fei Liang, Cong Shen, Feng Wu

arXiv:1707.05697v1Jul 18, 2017stat.MLcs.IT

TL;DR

The paper addresses decoding when noise correlation limits receiver performance. It proposes an iterative BP-CNN receiver with a new CNN loss function and reports improved decoding performance.

Problem
Receiver performance can be unsatisfactory when a designed channel code is not paired with a receiver that handles noise correlation.
Method
The proposed iterative BP-CNN architecture concatenates belief propagation with a CNN and uses a loss function designed to reduce residual noise power.
Results
Iterating between BP and CNN improves decoding SNR and results in better decoding performance.
Takeaways & Limitations
The iterative BP-CNN decoder provides a decoding structure for linear codes that addresses correlated noise.

Abstract

from arXiv · show

Inspired by recent advances in deep learning, we propose a novel iterative BP-CNN architecture for channel decoding under correlated noise. This architecture concatenates a trained convolutional neural network (CNN) with a standard belief-propagation (BP) decoder. The standard BP decoder is used to estimate the coded bits, followed by a CNN to remove the estimation errors of the BP decoder and obtain a more accurate estimation of the channel noise. Iterating between BP and CNN will gradually improve the decoding SNR and hence result in better decoding performance. To train a well-behaved CNN model, we define a new loss function which involves not only the accuracy of the noise estimation but also the normality test for the estimation errors, i.e., to measure how likely the estimation errors follow a Gaussian distribution. The introduction of the normality test to the CNN training shapes the residual noise distribution and further reduces the BER of the iterative decoding, compared to using the standard quadratic loss function. We carry out extensive experiments to analyze and verify the proposed framework. The iterative BP-CNN decoder has better BER performance with lower complexity, is suitable for parallel implementation, does not rely on any specific channel model or encoding method, and is robust against training mismatches. All of these features make it a good candidate for decoding modern channel codes.

I. INTRODUCTION

The paper addresses linear-code decoding under correlated channel noise, where model-based approaches can be complex and unreliable. It proposes an iterative BP-CNN receiver that exploits learned noise-correlation features and improves decoding performance with practical implementation benefits.

Motivation: Correlated channel noise can undermine otherwise well-designed channel codes, while strongly correlated noise makes joint-distribution-based BP optimization highly complex.Model-based approaches may also fail to obtain a well-behaved joint noise distribution.
Motivation: Existing deep-learning channel-decoding work did not address efficient decoding of linear codes under correlated channel noise.This gap motivates a learned receiver that avoids relying on a predefined channel model.
Iterative BP-CNN architecture: Iterating between BP and CNN gradually improves decoding SNR and results in better decoding performance.The CNN is trained to estimate channel noise while producing an output beneficial to the BP decoder.
Properties and contributions: The proposed decoder offers better decoding performance than standard BP with lower complexity, supports parallel computing, and adapts to different channel conditions and linear codes.Its CNN consists mostly of linear operations, and the method is described as data driven rather than dependent on predefined models.
Properties and contributions: The enhanced BP-CNN loss function combines residual-noise-power reduction with a Jarque-Bera normality test, and experiments evaluate the framework extensively.The normality component shapes the residual noise for improved compatibility with BP decoding.
Iterative BP-CNN architecture: The iterative BP-CNN architecture concatenates a standard BP decoder with a trained CNN and repeatedly processes received symbols between them.BP first estimates coded bits; subtracting estimated transmit symbols yields a noise estimate that the CNN refines using learned noise correlations.

II. RELATED WORKS

Prior work applies deep learning across communications and channel decoding, but efficient decoding of linear codes under correlated channel noise remains unaddressed.

CNNs have achieved strong results in image recognition, object detection, image denoising, and super-resolution, motivating their use for correlated-noise decoding.
Deep learning has been applied to channel decoding, MIMO detection, traffic recognition, channel auto-encoding, and model-free communication systems.
Neural channel decoders can reduce iterations or complexity, including trained Tanner-graph weights and hardware-friendly neural offset min-sum decoding.
Existing studies do not address channel decoding under correlated Gaussian noise, a complex setting that is difficult to analyze theoretically.
The paper therefore proposes an iterative BP-CNN receiver for channel decoding under correlated noise.

III. SYSTEM DESIGN

The system concatenates BP decoding with CNN-based noise estimation and feeds corrected observations back into BP. Training targets both accurate estimation and residual noise properties beneficial to subsequent decoding.

III. SYSTEM DESIGN: The iterative BP-CNN decoder concatenates a CNN with BP and repeatedly processes corrected observations to improve decoding.BP produces an initial estimate; CNN estimates channel noise, and the resulting corrected vector is fed back for another BP round.
III. SYSTEM DESIGN: The architecture is presented for LDPC codes but is readily applicable to other linear codes and does not depend on the specific correlation model.
III. SYSTEM DESIGN: After BP estimates the transmitted symbols, subtracting them from the received symbols yields an estimated channel noise vector with decoding-error contamination.
III. SYSTEM DESIGN: The CNN suppresses noise-estimation error by exploiting correlation in the channel noise, then produces a residual-noise estimate for BP updates.
III. SYSTEM DESIGN: Enhanced BP-CNN training jointly depresses residual-noise power and encourages a Gaussian residual distribution, unlike baseline quadratic-loss training.
III. SYSTEM DESIGN: Correlated additive noise matters in wireless and cellular systems, where fading and multi-user interference can create correlations relevant to decoding.

B. Why is CNN Useful for Channel Decoding?

CNN-based channel decoding is motivated by the analogy between noise recovery and image denoising. The decoder treats correlated channel noise as exploitable structure while retaining CNNs’ low-complexity denoising appeal.

B. Why is CNN Useful for Channel Decoding?: Channel decoding resembles image denoising because both recover an underlying signal from an additive noisy observation.Image denoising recovers X from Y = X + W; the decoder recovers channel noise from a noisy estimate.
B. Why is CNN Useful for Channel Decoding?: CNNs are attractive because prior image-denoising results combine strong restoration quality with low complexity.
B. Why is CNN Useful for Channel Decoding?: In this architecture, CNN output is used to recover channel noise and improve the subsequent channel-decoding process rather than merely extract features.
B. Why is CNN Useful for Channel Decoding?: The iterative BP-CNN decoder exploits correlations in channel noise analogously to CNN-based image denoising methods exploiting image structure.

C. Belief Propagation Decoding

Standard BP passes messages on a Tanner graph under a factored AWGN probability model. Correlated noise breaks that factorization, making direct incorporation of joint noise statistics potentially highly complex.

C. Belief Propagation Decoding: BP decoding iteratively passes messages between variable and check nodes in a Tanner graph, then determines bit values from final variable-node LLRs.
C. Belief Propagation Decoding: The paper introduces CNN-based noise estimation as an alternative processing stage following BP in the proposed architecture.
C. Belief Propagation Decoding: For AWGN channels, BP uses a joint bit-probability model combining channel-conditioned probabilities with parity-check indicators.
C. Belief Propagation Decoding: With correlated noise, the received-symbol-conditioned probability cannot be factored because the symbols are correlated.
C. Belief Propagation Decoding: Incorporating the joint distribution of correlated noise samples into standard BP may have very high complexity, especially under strong correlations.

D. CNN for Noise Estimation

The paper uses a one-dimensional CNN to estimate channel noise within the iterative BP-CNN receiver. Its enhanced loss jointly reduces residual-noise power and shapes residual noise toward a Gaussian distribution for subsequent BP decoding.

CNN motivation: CNN noise estimation is motivated by its feature-extraction and denoising capabilities in low-level restoration tasks.The proposed network adapts such architectures to one-dimensional channel-noise vectors.
CNN architecture: The proposed CNN takes a 1-D vector as input rather than a 2-D image and uses convolutional layers to construct feature maps.The architecture is specified by its layers, filter sizes, and feature-map counts.
CNN architecture: Pooling, dropout, and fully connected layers are omitted because the output preserves input dimensions, validation monitoring controls overfitting, and dense layers are costly for low-level tasks.The authors report no performance gain from dropout in their tests.
Loss function: The baseline loss minimizes residual-noise power, whereas the enhanced loss also shapes the residual-noise distribution for concatenation with BP.The CNN output affects the next BP iteration, so training considers the relationship between both stages.
Loss function: The enhanced loss combines residual-noise power with a Jarque-Bera normality term that measures Gaussian conformity, balanced by λ.Its skewness and kurtosis components are derivable and simple to train, although the Jarque-Bera test is not optimal.
Loss function: Experiments report that the new loss provides a desirable output, while selecting λ analytically remains difficult.The parameter is therefore chosen through simulations.

2) Generating the training data:

Training data are generated by simulating coded transmissions through correlated channels and decoding them with BP to obtain estimated noise inputs. The training SNR range must balance insufficient and excessive BP-decoding errors, while practical deployment relies on offline training and stored networks.

Data generation: Estimated noise data are generated from encoded, BPSK-mapped bits passed through simulated channel interference and BP decoding.The process also uses AWGN data and a channel correlation matrix to construct training samples.
Channel conditions: Training data span multiple channel SNRs because SNR controls BP-decoding errors and therefore the CNN input.The SNR conditions are represented by the set Γ.
Channel conditions: Very good channels provide too few BP errors for robust noise-feature learning, whereas very bad channels produce errors that mask channel-noise features.The authors analyze this training-data problem experimentally.
Practical deployment: CNN training requires substantial data and computational complexity, but these requirements are handled largely offline using powerful devices and mass storage.The resulting trained network can be stored in on-device memory for online use.
Practical deployment: Known noise-correlation functions allow training data to be generated for representative communication-system use cases.The paper frames this as a way to address dependence of CNNs on training data.

F. Design Summary

The design summary treats iterative BP-CNN decoding as a configurable architecture whose iterations, CNN structure, BP iterations, and loss weighting determine performance–complexity trade-offs. Experiments use a fixed LDPC setup and evaluate BER across correlated and uncorrelated channels.

Architecture parameters: The iterative architecture is parameterized by total iterations K, CNN layers and feature maps, filter sizes, and BP-decoder iterations.It can be unfolded into an open-loop BP-CNN1-BP-CNN2-...-CNNx-BP framework.
Architecture parameters: The closed-loop design stores one CNN, whereas the more general open-loop framework uses x CNNs with higher training and storage demands.Serially concatenated CNNs may depend on earlier networks during training.
Experimental design: The enhanced design requires λ and training-channel conditions to be selected carefully, and experiments provide guidance for these parameters.The study evaluates their influence on decoding performance.
Performance evaluation: With equal total BP iterations, BP-CNN is tested as BP(25)-CNN-BP(25), while standard BP is tested with BP(50).The proposed design uses one BP–CNN iteration in this comparison.
Performance evaluation: At η = 0.8, BP-CNN improves decoding by approximately 3.5dB at BER=10−4, and this gain is not matched by additional standard-BP iterations.At η = 0.5 the gain is smaller, while at η = 0 the method performs similarly to standard BP.
Performance evaluation: The enhanced decoder outperforms baseline BP-CNN, with gains increasing monotonically as correlation η increases.The authors attribute this trend to greater opportunity for CNNs to extract noise features.

2) BP-CNN achieves performance gain with lower complexity:

BP-CNN improves decoding performance at lower comparable complexity, especially under strong noise correlation. The enhanced loss matters because its normality term shapes residual noise, while λ must balance distributional shaping against residual-noise power.

Complexity comparison: A CNN runtime is roughly equivalent to two BP iterations, making BP(5)-CNN-BP(5) approximately as complex as standard BP(12).The comparison uses matched TensorFlow environments and shared computation resources.
Complexity comparison: BP(5)-CNN-BP(5) has comparable performance to BP(50) with much lower complexity.BP(12) is used as the fairer good-but-not-saturated comparison at approximately matched complexity.
Complexity comparison: At approximately equal complexity, both baseline and enhanced BP(5)-CNN-BP(5) outperform BP(12).The gain is around 3dB under strong correlation and 0.1 to 0.5dB under moderate correlation.
Normality-test ablation: Small λ values cannot guarantee Gaussian residual noise, while large λ values cannot sufficiently reduce residual-noise power.Thus λ affects enhanced BP-CNN performance and is selected through simulations rather than analytic optimization.
Normality-test ablation: Including the normality test makes the residual-noise distribution more suitable for concatenation with BP.The comparison uses empirical residual-noise distributions from enhanced and baseline BP-CNN.
Hyperparameter choice: For η = 0.8, λ = 0.1 performs best, whereas λ = 10 performs best for η = 0.5.The paper explains that stronger correlation permits more error removal, favoring greater emphasis on residual-noise power.

4) Multiple iterations between CNN and BP further improve the performance:

Iterating between BP and CNN further improves decoding performance, with gains eventually saturating as the CNN reaches its capacity.

Multiple BP-CNN iterations further improve decoding performance.
0.7dB improvement at BER=10−4 occurs with two iterations versus one when η = 0.8.
After four BP-CNN iterations, additional performance improvement becomes insignificant.
The performance saturation is attributed to the CNN reaching its maximum capacity to depress residual noise power.

5) BP-CNN is robust under different correlation models:

The BP-CNN decoder maintains performance gains across correlation models, while complexity allocation and training-data diversity affect its effectiveness.

The decoder achieves significant performance gains with pink noise, supporting operation beyond the original correlation model.The pink-noise experiment uses λ = 0.1 for training.
Increasing BP complexity improves performance when the underlying CNN structure is unchanged.
When correlation is strong or moderate, increasing CNN complexity is more effective than increasing BP complexity.BP(5)-CNN(HP)-BP(5) outperforms the lower-complexity CNN alternative despite lower overall complexity than BP(10)-CNN(LP)-BP(10).
Under AWGN channels, assigning more complexity to BP is more effective because the channel has no feature for CNN extraction.
Training data generated across a wide SNR range generally performs better than data generated at a single SNR.For η = 0.5, training at {0, 0.5, 1, 1.5, 2, 2.5, 3}dB outperforms training at 0dB or 3dB.
Selecting training channel conditions remains difficult, so the authors suggest multiple conditions to increase data diversity.

V. CONCLUSIONS AND FUTURE DIRECTIONS

The paper concludes that iterative BP-CNN decoding addresses correlated channel noise through alternating BP and CNN processing, with simulations supporting its effectiveness. Future work includes more flexible architectures and improved loss functions.

The proposed iterative BP-CNN structure is designed to handle correlated channel noise.
BP estimates coded bits and indirectly estimates noise, while CNN learns noise correlation to remove BP estimation errors.
Extensive simulations show the effectiveness of the proposed iterative BP-CNN decoder.
Future directions: The loss function is not completely equivalent to system performance, motivating improved loss-function design.
Future directions: An open-loop generalization can vary BP iterations and CNN structures across stages to allocate complexity more flexibly.The proposed form is BP(n1)-CNN1-BP(n2)-CNN2-...-BP(nx)-CNNx-BP(nx+1).