PaperlayerView exact revision on arXiv

Source-linked AI summary

Trainable Communication Systems: Concepts and Prototype

Sebastian Cammerer, Fayçal Ait Aoudia, Sebastian Dörner, Maximilian Stark, Jakob Hoydis, Stephan ten Brink

arXiv:1911.13055v2cs.ITeess.SP

TL;DR

The paper addresses the difficulty of optimizing neural communication systems for practical BMD receivers and develops BMI-trained autoencoders with differentiable IDD and code optimization. The resulting systems learn constellation shaping, labeling, and demapping jointly, outperform conventional baselines in simulations and realistic wireless experiments, and support further gains through code design.

  • Problem

    Symbol-wise autoencoder training does not directly provide suitable labeling for practical BMD receivers, while mutual information is not an achievable BMD rate.

  • Method

    The authors train bit-wise autoencoders on BMI, jointly optimize constellation shaping and labeling, and integrate learned LLR demapping with differentiable IDD and LDPC coding.

  • Results

    Approximately 1.3 dB improvement over a 256-QAM baseline with an 802.11n LDPC code was achieved at the same communication rate in a realistic setup.

  • Takeaways & Limitations

    BMI-trained systems provide practical BMD integration, while IDD and LDPC code optimization offer additional performance improvements across simulations and over-the-air experiments.

Abstract

from arXiv · show

We consider a trainable point-to-point communication system, where both transmitter and receiver are implemented as neural networks (NNs), and demonstrate that training on the bit-wise mutual information (BMI) allows seamless integration with practical bit-metric decoding (BMD) receivers, as well as joint optimization of constellation shaping and labeling. Moreover, we present a fully differentiable neural iterative demapping and decoding (IDD) structure which achieves significant gains on additive white Gaussian noise (AWGN) channels using a standard 802.11n low-density parity-check (LDPC) code. The strength of this approach is that it can be applied to arbitrary channels without any modifications. Going one step further, we show that careful code design can lead to further performance improvements. Lastly, we show the viability of the proposed system through implementation on software-defined radios (SDRs) and training of the end-to-end system on the actual wireless channel. Experimental results reveal that the proposed method enables significant gains compared to conventional techniques.

I. INTRODUCTION

The paper develops trainable communication systems that optimize bit-wise mutual information for practical BMD receivers, jointly learning constellation geometry, labeling, and demapping. It further introduces differentiable iterative demapping and decoding and validates the approach in simulations and wireless experiments.

  • Bit-wise optimization: Training on BMI provides a BMD-achievable objective while jointly optimizing constellation shaping and bit labeling.The mapper assigns each bit vector to a constellation point, while the demapper produces bit-level outputs suitable for decoder integration.
  • Iterative decoding: The learned demapper can be interfaced with belief-propagation channel decoding, enabling a differentiable neural IDD architecture.The proposed framework is designed to incorporate prior bit information from the decoder into subsequent demapper iterations.
  • Practical validation: The approach is intended to apply without modification to arbitrary channel models and is demonstrated through software-defined-radio training on an actual wireless channel.The authors present this as evidence of the method’s universality and practical viability.
  • Motivation: Symbol-wise autoencoders require a difficult post-training labeling step before their learned constellations can be used with practical BMD receivers.For non-grid constellations, finding the optimal labeling has 2^m! possibilities, neglecting symmetries.
  • Bit-wise autoencoder: The bit-wise autoencoder maps input bit vectors to constellation points and outputs one logit per bit, which can be interpreted as an LLR for decoding.Bit probabilities are obtained by applying sigmoid functions element-wise to the demapper logits.

B. Simulation setup

The system is evaluated on AWGN using neural mapper and demapper components, with SNR-dependent constellations and an LDPC belief-propagation decoder. Training uses sampled bit vectors and varying SNR, while evaluation compares BER against conventional baselines.

  • Each transmitted bit vector produces a complex symbol, while the demapper converts each received sample into m log-likelihood ratios for LDPC decoding.The complete codeword’s LLRs are concatenated and passed to a BP decoder.
  • The mapper generates an SNR-dependent constellation, normalizes its average symbol energy, and selects a point using the input bit vector’s one-hot representation.The constellation contains 2^m complex-valued symbols, and normalization ensures E_x = 1.
  • Without SNR feedback, the same architecture can be trained at fixed SNR, at the cost of slightly reduced BER performance.
  • Training uses Adam, a batch size of 500, and a learning rate decreased from 10^-3 to 10^-5, with SNR sampled uniformly over a 4 dB range.The channel encoder and decoder are omitted during training; evaluation uses 40 BP decoder iterations.
  • The evaluation compares BER for PSK at m = 3 and QAM at m = 4, 6, and 8 against the bit-wise autoencoder.

C. Results

The bit-wise autoencoder outperforms conventional PSK/QAM baselines and learns constellations tailored to BMI and BER rather than symbol-wise error. Its gains and geometry differ from symbol-wise training, especially when labeling is difficult.

  • 0.8 dB gain is achieved for m = 8, compared with 0.3 dB for m = 3, as the bit-wise autoencoder outperforms the PSK/QAM baselines.The comparisons use PSK for m = 3 and QAM for m = 4, 6, and 8.
  • BMI training produces an m = 4 constellation that differs significantly from the constellation learned with symbol-wise categorical cross-entropy.Using the symbol-wise constellation in BMD additionally requires heuristic labeling.
  • At low SNR, bit-wise optimization clusters points differing in one bit, improving overall achievable information rate while weakening that bit’s reliability.The same clustering creates unavoidable symbol-metric confusion and can degrade symbol-wise performance.
  • Symbol-wise training improves SER but degrades achievable BER, while heuristic labeling is not necessarily optimal for BMD.Figure 5 compares random and heuristic labels with QAM using Gray labeling and the bit-wise autoencoder.

III. BIT-WISE ITERATIVE AUTOENCODER

The bit-wise autoencoder is extended with a differentiable iterative demapping and decoding receiver. The demapper and LDPC decoder exchange extrinsic information so each can refine the other’s soft estimates.

  • IDD applies the Turbo Principle by combining soft demapping with channel decoding.
  • Each LDPC codeword is divided into s bit vectors, producing s autoencoder transmissions whose demapper outputs are processed by the iterative receiver.
  • The demapper receives channel samples and decoder-derived prior LLRs, then sends extrinsic information to the decoder.At the first iteration, the prior information is the null vector.
  • The decoder uses the error-correcting code to compute improved LLRs, which are fed back as prior information to refine demapping.This feedback compensates for information loss caused by bit-metric decoding.
  • The end-to-end system can be trained with channel decoding and IDD for any channel model.
  • The described implementation assumes LDPC codes with belief-propagation decoding, although the approach can generalize to other coding schemes and decoding algorithms.

E 11 end

The unfolded IDD receiver forms a differentiable feedforward neural network that can be trained end to end. It jointly optimizes the constellation, labeling, and demapper across iterative decoding.

  • The IDD algorithm computes demapper extrinsic information, concatenates it across symbols, and performs belief-propagation updates iteratively.Decoder-to-demapper information is initialized and updated across iterations.
  • Loop unrolling converts the iterative receiver into a feedforward neural network containing only differentiable operations.The first, intermediate, and final iterations form the unfolded computational graph.
  • End-to-end training jointly optimizes the constellation, labeling, and demapper for the considered channel and iteration count.The same trainable demapper can be reused across iterations, avoiding an increase in parameter count.
  • The IDD autoencoder operates on complete codewords rather than individual complex baseband symbols.Its bit loss aggregates information over the bits in a codeword and the received baseband samples.
  • Training uses binary cross-entropy at the demapper output over all iterations because direct end-to-end loss minimization produced poor performance.The posterior estimates are obtained by applying the sigmoid function to the demapper logits.

B. Simulation setup and results

The simulations evaluate iterative bit-wise autoencoders and code design on AWGN channels using PSK/QAM baselines and LDPC constructions. IDD improves both baseline and learned systems, while the autoencoder provides substantial gains at high modulation orders.

  • Simulation setup: The iterative bit-wise autoencoder is trained through a differentiable IDD receiver with I = 40 iterations.The same demapper weights are used at every iteration.
  • Simulation results: IDD improves the BER performance of both standard PSK/QAM modulations and the bit-wise autoencoder.The comparison covers m = 3, 4, 6, and 8 bits per symbol.
  • Simulation results: The bit-wise autoencoder with IDD achieves significant gains over standard modulations.The baseline is PSK for m = 3 and QAM otherwise.
  • Simulation results: At modulation orders 64 (m = 6) and 256 (m = 8), the autoencoder without IDD achieves lower BERs than the baseline with IDD.This comparison is reported for high modulation orders.
  • Code design setup: The code-design section models LDPC codes with Tanner graphs, parity-check matrices, and variable- and check-node degree profiles.The conventional design approach optimizes degree distributions using EXIT-chart analysis.

A. EXIT analysis

EXIT analysis characterizes the demapper’s extrinsic information as a function of a priori information and supports code design for iterative decoding. The analysis uses Monte Carlo estimation and Gaussian-distributed a priori LLRs.

  • Extrinsic information: The IDD demapper must exclude a priori information from its output so that only new extrinsic information is passed between receiver components.The a priori contribution is explicitly subtracted because directly maximizing output mutual information violates the iterative-receiver condition.
  • Assumptions: EXIT analysis assumes Gaussian-distributed decoder LLRs, an assumption considered valid for sufficiently many iterations and sufficiently long codewords.The analysis also uses a pseudo-random scrambling sequence to ensure channel symmetry during code design.
  • EXIT modeling: EXIT analysis replaces the demapper’s a priori LLRs with Gaussian-distributed LLRs whose mean is determined by their mutual information.The approximation makes the code-design analysis simpler to simulate.
  • EXIT modeling: The demapper EXIT characteristic T(.) gives the extrinsic mutual information produced for a given a priori mutual information.It describes the additional information obtained from the channel output by the iterative demapper.
  • EXIT estimation: The demapper transfer function is estimated with Monte Carlo methods because a closed-form solution is difficult to obtain for the trainable receiver.The estimate uses measured or simulated extrinsic LLRs and sufficiently many samples.

B. Code design method

The code-design method optimizes LDPC degree profiles against the trained demapper’s EXIT characteristic and then constructs a finite code. The optimized code maintains an open EXIT tunnel and improves BER relative to the 802.11n baseline.

  • Degree-profile optimization: LDPC degree-profile optimization maximizes design rate while requiring belief-propagation decoding to converge to zero average bit error probability.The optimization uses a check-concentrated distribution and constrains the variable-node degree profile.
  • SNR targeting: The optimization is solved repeatedly across SNR values to find the lowest SNR supporting a target rate.The demapper transfer function is numerically estimated for each fixed SNR.
  • Finite-code construction: After optimizing the ensemble, a single code is constructed with progressive edge growth to maximize graph girth and reduce short cycles.The baseline is the conventional 802.11n code with n = 1944 bits.
  • EXIT validation: The simulated decoding trajectory tracks the predicted EXIT curves and maintains an open tunnel throughout decoding.This verifies the effectiveness of the optimized code for the trained IDD receiver.
  • BER results: Approximately 0.1 dB and 0.2 dB BER gains over the baseline are achieved for m = 6 and m = 8, respectively.The comparison uses optimized codes against the 802.11n code of length n = 1944 on AWGN channels.

V. OVER-THE-AIR EXPERIMENTS

Training over an actual wireless channel is motivated by performance loss caused by mismatch between a channel model and the deployed channel.

  • Channel-model training can lose significant performance after deployment on the actual channel.The reported loss is attributed to unavoidable mismatch between the modeled and actual channels.

A. Training over the actual channel

The end-to-end system is trained over an actual channel by alternating conventional receiver updates with perturbation-based mapper updates that approximate transmitter gradients without channel differentiation.

  • Training alternates between conventional receiver training and mapper training over the actual channel.Receiver training does not require differentiating through the channel.
  • Random perturbations added to transmitter outputs enable gradient approximation despite the absence of a channel model.The approach is inspired by reinforcement learning.
  • The perturbation variance controls a tradeoff between gradient-approximation accuracy and estimator variance.Reducing σ_w improves approximation accuracy but increases variance and slows convergence.
  • The mapper sends perturbed symbols through the channel, then uses receiver-generated LLRs to estimate the transmitter loss gradient.A reliable training-only link returns the LLRs to the mapper.

B. Experimental results

Over-the-air experiments train an OFDM autoencoder with an 802.11n LDPC code and show gains over QAM baselines, with further improvements from code optimization and channel-adapted demapping.

  • The over-the-air system used two USRPs at 2.35 GHz with 15.94 MHz bandwidth in a static indoor office environment.The OFDM setup used 64 subcarriers, including 50 for data transmission, with a cyclic prefix ratio of 1/8.
  • The autoencoder used per-subcarrier MMSE equalization before neural-network demapping, without compensating for other potential impairments.The neural network did not access the cyclic prefix or other subcarrier channel conditions.
  • A single fixed constellation was trained over an SNR range centered around the LDPC waterfall region because precise pre-transmission SNR sounding was unavailable.The current SNR was not provided to the encoder for the over-the-air results.
  • Approximately 1.0 dB separates the autoencoder from QAM baselines over the actual channel, with code optimization adding 0.2 dB for m = 6 and 0.4 dB for m = 8.The reported curves use IDD with 80 iterations and the same-length 802.11n code for the baseline and optimized system.
  • The gains are attributed to optimized constellations, a demapper adapted to the actual channel, and a matched LDPC code.The learned demapper can provide LLR estimates closer to the actual posterior probabilities than an AWGN-MAP demapper on the OTA channel.

VI. CONCLUSION

The paper validates training-based optimization of a coded point-to-point physical layer, combining BMI training, iterative demapping and decoding, and over-the-air learning. The trained system outperforms a 256-QAM baseline by approximately 1.3 dB at the same communication rate, while training overhead makes the approach particularly attractive for fixed wireless or optical channels.

  • BMI training enables seamless integration of the learned system with practical bit-metric decoding receivers.The approach jointly supports physical-layer optimization and an outer channel code.
  • Iterative demapping and decoding are integrated into the end-to-end learning procedure, and LDPC code optimization can provide additional gains.Code optimization is applied after the system has been trained.
  • Approximately 1.3 dB improvement over a 256-QAM baseline with an 802.11n LDPC code is achieved at the same communication rate.The result was demonstrated in a realistic setup using simulations and over-the-air experiments, including optimization from scratch on an actual wireless channel.
  • The required training overhead makes the approach particularly attractive for fixed wireless or optical channels.The paper identifies probabilistic shaping, multiuser communication, and fast online learning as future directions.
Loading 1911.13055v2…