Source-linked AI summary
End-to-End Learning of Communications Systems Without a Channel Model
Fayçal Ait Aoudia, Jakob Hoydis
TL;DR
Conventional autoencoder-based end-to-end communication learning requires a differentiable channel model. The paper alternates supervised receiver training with reinforcement-learning transmitter training, achieving similar performance to fully supervised methods on AWGN and RBF channels while converging at different rates.
Problem
End-to-end autoencoder training requires a differentiable channel model, although actual communication channels are generally observable only through inputs and outputs.
Method
The method independently optimizes transmitter and receiver functions by alternating supervised receiver training with RL-based transmitter training from receiver-provided losses.
Results
The alternating method achieves similar performance to fully supervised training on AWGN and RBF channels, converging slower on AWGN but faster on RBF.
Takeaways & Limitations
The approach demonstrates end-to-end training from pure observations without knowledge of the underlying channel model.
Takeaways & Limitations
Training currently requires an additional reliable channel to provide receiver losses to the transmitter.
Abstract
from arXiv · showhide
The idea of end-to-end learning of communications systems through neural network -based autoencoders has the shortcoming that it requires a differentiable channel model. We present in this paper a novel learning algorithm which alleviates this problem. The algorithm iterates between supervised training of the receiver and reinforcement learning -based training of the transmitter. We demonstrate that this approach works as well as fully supervised methods on additive white Gaussian noise (AWGN) and Rayleigh block-fading (RBF) channels. Surprisingly, while our method converges slower on AWGN channels than supervised training, it converges faster on RBF channels. Our results are a first step towards learning of communications systems over any type of channel without prior assumptions.
I. INTRODUCTION
End-to-end learning jointly optimizes neural-network transmitters and receivers for communication performance, but conventional autoencoder training requires a differentiable channel model. This paper investigates reinforcement learning to train from observations over unknown channels.
- End-to-end learning represents the transmitter and receiver as neural networks and jointly optimizes them for a specific performance metric and channel model.
- The main practical obstacle is that supervised autoencoder training requires the channel model or instantaneous channel-transfer gradient, which is generally unavailable for black-box channels.
- Receiver-only fine-tuning on measured data leaves the transmitter unchanged, resulting in sub-optimal performance.
- The paper investigates reinforcement learning as a way to estimate loss gradients with respect to transmitter actions without knowing the channel model.
- The proposed approach targets end-to-end optimization without compartmentalizing transmitter and receiver blocks, motivated by the complexity of jointly optimizing conventional components.
A. Training Process Overview
The alternating training algorithm independently optimizes transmitter and receiver functions by repeatedly training the receiver and then the transmitter. Its convergence is expected intuitively but is not formally established in the supplied passage.
- Unlike single-function approaches combining transmitter, channel, and receiver, the method uses two independently optimized parametric functions and requires no channel model.
- Each main iteration alternates supervised receiver training with RL-based transmitter training until a stopping criterion is satisfied.
- The receiver is improved with fixed transmitter parameters, then the transmitter is improved with fixed receiver parameters.
- Formal convergence analysis of the alternating approach remains under investigation.
- SGD approximates the loss gradient using minibatches and is used in both phases to optimize transmitter and receiver parameters.
B. Receiver Training
Receiver training is supervised: transmitted training messages pass through the channel, and the receiver updates its parameters using cross-entropy loss.
- Receiver training uses labeled messages, making it a supervised learning task.
- The transmitter encodes a minibatch of messages into N channel symbols and sends them over the channel.
- The receiver processes altered symbols, outputs a probability distribution over the message set, and updates its parameters with SGD on cross-entropy loss.
C. Transmitter Training
Transmitter training treats message encoding as a stochastic policy whose symbols are optimized using per-example losses returned by the receiver. Training therefore requires a reliable auxiliary channel for loss feedback.
- The transmitter generates channel symbols to minimize a scalar loss supplied by the receiver, forming the RL-based training phase.
- The message set is the RL state space, complex channel symbols are actions, and policy randomization enables exploration during transmitter training.
- The transmitter updates its parameters using received losses, while the number of inner training iterations may be fixed or controlled by a stopping criterion.
- During transmitter training, a stochastic policy is sampled to perturb encoded symbols before they are sent through the channel.
- The receiver computes per-example losses from its output probabilities and the sent messages, then returns those losses to the transmitter.
III. GENERIC TRANSMITTER AND RECEIVER ARCHITECTURES
The paper uses separate differentiable transmitter and receiver functions, implemented as neural networks, for communications-system learning. The transmitter normalizes complex channel symbols, while the receiver converts received signals and outputs message probabilities for hard decoding.
- The alternating training algorithm applies to any pair of differentiable parametric transmitter and receiver functions, implemented here as feedforward neural networks.Each network is a parametric function composed of successive layers.
- The transmitter converts real-valued intermediate outputs into N complex channel symbols and normalizes them so average energy per symbol or message is 1.This architecture addresses complex baseband transmission and power constraints.
- The receiver converts an N-dimensional complex received vector into 2N real scalars, then performs soft detection by outputting a probability distribution over M messages.A softmax output supports probability-based detection, followed by choosing the highest-probability message.
IV. EVALUATION
The evaluation compares the proposed alternating training scheme with fully supervised training, whose channel model must be differentiable and approximately match the communication channel.
- The proposed alternating scheme is compared with the fully supervised approach of.The comparison concerns communications-system performance.
- Fully supervised training represents the transmitter, channel, and receiver as one deep neural network and therefore requires a differentiable channel model.The model should at least approximately match the channel on which the system is planned to communicate.
SNR “ E
The experiments define SNR using expected complex-symbol energy and noise variance, while transmitter exploration adds a complex Gaussian perturbation and preserves average symbol energy.
- The transmitter is normalized so E=1, leading to SNR = 1/σ2.Here, σ2 denotes the noise variance.
- RL exploration adds a zero-mean complex normal perturbation w to the transmitter output to produce the channel input xp.The perturbation is governed by a fixed exploration parameter σπ.
- The transmitter RL policy is a Gaussian policy whose mean is the transmitter output, with policy parameters relevant only during training.The text identifies the policy parameter vector as empty for the specified setting.
- The transmitter and receiver functions are implemented as deep neural networks, and training uses Adam with SNRs of 10 dB for AWGN and 20 dB for RBF channels.The experiments use M=256 messages and N=4 channel uses.
A. Transmitter and Receiver Architectures
The implemented system uses dense feedforward neural networks with transmitter normalization and receiver softmax decoding. For RBF channels, the receiver architecture incorporates channel-response estimation before message classification.
- Transmitter and Receiver Architectures: Dense layers compute affine transformations followed by elementwise activation functions, with weight matrices and bias vectors as trainable parameters.Nonlinear activations allow neural networks to approximate a wide range of functions.
- Transmitter and Receiver Architectures: The transmitter uses an M × M embedding with ELU activations, followed by 2N linear outputs converted into N complex symbols and normalized.The normalization is the architecture shown for the transmitter.
- Transmitter and Receiver Architectures: The receiver converts N complex symbols into 2N real symbols and ends with an M-unit softmax layer producing a probability distribution over M messages.For AWGN, one dense ReLU hidden layer is used.
- Transmitter and Receiver Architectures: For RBF channels, the first two receiver layers estimate the channel response before the received signal is divided by that estimate and classified.The final two layers generate a probability distribution over M.
B. Evaluation Results
The alternating method converges more slowly than supervised training on AWGN but faster and with less variance on RBF, while achieving similar final error-rate performance.
- On AWGN, supervised training converges faster because exact channel gradients provide finer transmitter feedback than noisy loss-based gradient estimates.This advantage requires a differentiable channel model that approximates the real communication channel.
- After approximately 400 training iterations on AWGN, no significant performance difference is observed.
- On RBF, alternating training converges faster and with significantly less variance than the supervised approach.The supervised gradient is directly affected by the random channel response, whereas alternating training uses per-example losses less affected by that response.
- The alternating method without a channel model achieves similar performance to fully supervised training on both AWGN and RBF channels.
V. CONCLUSION AND ONGOING WORK
The proposed method achieves performance similar to fully supervised training without requiring a mathematical channel model, but currently needs a reliable feedback channel during training.
- The method achieves similar performance to the fully supervised approach while requiring no mathematical model of the channel.
- The algorithm can therefore be applied to any channel type without prior channel analysis.
- Current training requires an additional reliable channel to feed losses back from the receiver to the transmitter.The authors are investigating training without such a dedicated feedback channel.
- Future work includes advanced reinforcement-learning techniques to potentially increase convergence speed and extensions to source coding, reconstruction, and classification tasks.
APPENDIX BACKGROUND ON REINFORCEMENT LEARNING
The reinforcement-learning background frames training as policy optimization from per-example losses, using stochastic exploration and parameter-gradient estimation under regularity conditions.
- Reinforcement-learning formulation: An RL agent takes actions according to a policy and receives per-example losses from its environment.The expected loss conditional on state and action is treated as unknown, and the policy is optimized to minimize it.
- Reinforcement-learning formulation: A stochastic policy defines a probability distribution over actions conditional on the agent’s state.Stochasticity enables exploration of the environment during training.
- Policy-gradient methods: Policy-gradient methods optimize a parametric policy by estimating the gradient of the loss with respect to its parameters.
- Policy-gradient methods: Exchanging integration and differentiation requires regularity conditions.