Model-free Training of End-to-end Communication Systems

Fayçal Ait Aoudia; Jakob Hoydis

Source-linked AI summary

Model-free Training of End-to-end Communication Systems

Fayçal Ait Aoudia, Jakob Hoydis

arXiv:1812.05929v3Dec 14, 2018cs.ITcs.AIstat.ML

TL;DR

Conventional end-to-end autoencoder training requires a differentiable channel model, which is unavailable for unknown channels and non-differentiable components. The paper introduces an alternating model-free algorithm using the true receiver gradient and an approximate transmitter gradient. It matches model-based training across several channels and tasks and demonstrates competitive hardware performance over coaxial and wireless channels.

Problem
End-to-end autoencoder training requires a differentiable channel model or channel gradient, which practical black-box channels and non-differentiable transceiver components may not provide.
Method
The method alternates receiver training with the true loss gradient and transmitter training with an approximate gradient obtained by relaxing the channel input to a random variable.
Results
Model-free training achieves the same performance as model-based training across several simulated channels and tasks and yields competitive performance over actual wireless and coaxial channels.
Takeaways & Limitations
The algorithm enables direct training of autoencoder-based communication systems from observations without channel-model knowledge, including hardware experiments over actual channels.

Abstract

from arXiv · show

The idea of end-to-end learning of communication systems through neural network-based autoencoders has the shortcoming that it requires a differentiable channel model. We present in this paper a novel learning algorithm which alleviates this problem. The algorithm enables training of communication systems with an unknown channel model or with non-differentiable components. It iterates between training of the receiver using the true gradient, and training of the transmitter using an approximation of the gradient. We show that this approach works as well as model-based training for a variety of channels and tasks. Moreover, we demonstrate the algorithm's practical viability through hardware implementation on software-defined radios where it achieves state-of-the-art performance over a coaxial cable and wireless channel.

I. INTRODUCTION

End-to-end autoencoder learning jointly optimizes transmitter and receiver networks for a channel and performance metric, but conventional training requires a differentiable channel gradient. This motivates a model-free alternating method that trains the receiver with the true gradient and the transmitter with an approximate gradient, enabling evaluation across simulated and actual channels.

I. INTRODUCTION: End-to-end learning represents the transmitter and receiver as neural networks within an autoencoder optimized for a specific performance metric and channel model.The transmitter maps messages to channel symbols, while the receiver outputs message probabilities.
I. INTRODUCTION: Conventional autoencoder training requires the channel transfer-function gradient, which is unavailable for black-box channels and blocked by non-differentiable components such as quantization.Only channel inputs and outputs may be observable in practical systems.
I. INTRODUCTION: The proposed model-free approach relaxes the channel input to a random variable and alternates receiver training with true gradients and transmitter training with an approximate gradient.This removes the requirement for channel-model knowledge and allows training from observations alone.
I. INTRODUCTION: Across AWGN, Rayleigh block-fading, and simplified fiber-optic channels, model-free and model-based training achieve identical performance, while learned systems outperform QPSK in the reported channel settings.On AWGN the learned system approaches Agrell, and on Rayleigh block-fading it outperforms both QPSK and Agrell.
I. INTRODUCTION: Hardware experiments over wireless and coaxial channels show competitive performance against a well-designed baseline, with the coaxial result also outperforming Agrell.On wireless, prior equalization improves performance relative to Agrell, whereas the no-equalization result does not.
I. INTRODUCTION: The method addresses limitations of component-wise communication-system design by jointly optimizing transmitter and receiver functions for end-to-end performance.Component-wise optimization is not guaranteed to achieve the best possible performance and can be computationally intractable when components are jointly optimized directly.

A. Gradient of the receiver

Receiver training can use sampled channel outputs without knowing the channel model, while transmitter-gradient approximation relies on relaxing the channel input and has stated theoretical conditions.

The receiver gradient can be estimated by sampling channel outputs, so receiver training does not require knowledge of p(y|x).The estimator is valid for i.i.d. training samples.
The transmitter output is relaxed from a Dirac-distributed input to a distribution with standard deviation σ > 0.This relaxation defines the associated substitute loss.
Sampling the relaxed channel-input distribution yields a transmitter-gradient estimate that does not require channel differentiability or model knowledge.The estimate uses the channel distribution through sampling rather than differentiating it.
The approximation can be interpreted as estimating the unknown channel gradient, with the transmitter acting as an agent interacting with the channel–receiver environment.The reinforcement-learning interpretation treats transmitted symbols as actions and per-example losses as penalties.
Under stated conditions, the substitute transmitter gradient can approximate the true gradient with arbitrarily small error.The paper notes that the theoretical result assumes p(y|x) is differentiable with respect to x.
The sampling estimators remain computable for non-differentiable channels, but theoretical guarantees in that case are left open.The paper distinguishes computability of the estimators from guarantees for the non-differentiable case.

IV. TRAINING END-TO-END COMMUNICATION SYSTEMS

The paper trains transmitter and receiver with an alternating model-free procedure, contrasting it with model-aware backpropagation that requires a differentiable channel model.

The alternating algorithm applies to pairs of differentiable parametric transmitter and receiver functions, implemented here as neural networks.The method is called model-free because it does not require a channel model.
Model-aware training combines transmitter, channel model, and receiver into one neural network optimized by usual backpropagation.This approach requires knowledge of a differentiable channel model.

A. Generic transmitter and receiver architectures

The system uses neural transmitter and receiver architectures with output constraints and alternating updates that separately optimize each side while holding the other fixed.

A. Generic transmitter and receiver architectures: The transmitter normalizes its final output to satisfy average-energy or other hardware-dependent constraints.Non-differentiable transmitter operations such as quantization can be treated as part of the channel.
A. Generic transmitter and receiver architectures: The receiver produces a probability vector over M through a softmax layer before selecting the highest-probability message.Non-differentiable receiver components can likewise be modeled as part of the channel.
IV. TRAINING END-TO-END COMMUNICATION SYSTEMS: Each training iteration alternates receiver training with true gradients and transmitter training with an approximate gradient.During either phase, the other network’s parameters remain fixed.
IV. TRAINING END-TO-END COMMUNICATION SYSTEMS: The alternating process repeats until a stopping criterion is met.The criterion may be a fixed iteration count or a specified period without loss decrease.
IV. TRAINING END-TO-END COMMUNICATION SYSTEMS: Transmitter and receiver share a sequence of training examples, which can be synchronized using pseudorandom generators initialized with the same seed.Both phases use gradient descent, SGD, or variants; receiver and transmitter gradients are estimated separately.

C. Receiver training

Receiver training sends encoded message minibatches through the channel, computes probability-based losses from received symbols, and updates the receiver without relaxing transmitter outputs.

C. Receiver training: Receiver training generates a minibatch of messages and encodes each into 2N real channel symbols.The encoded symbols form a BR-by-2N matrix X before transmission.
C. Receiver training: The transmitter outputs are sent directly through the channel because relaxation is unnecessary for estimating the receiver gradient.The receiver then obtains altered symbols and produces a probability vector over messages.
C. Receiver training: Receiver training computes per-example losses from the receiver’s probability outputs and the transmitted messages.The pseudocode identifies this phase as TRAINRECEIVER.
C. Receiver training: Transmitter training instead samples relaxed outputs, sends them through the channel, and returns per-example losses to the transmitter over a reliable feedback link.This process is illustrated in the transmitter-training phase of Fig. 3.

V. EVALUATION BY SIMULATIONS

The simulations evaluate model-free training under AWGN and RBF channels, using Gaussian relaxation noise and fixed optimization settings.

Smaller σ improves gradient approximation but increases estimator variance and slows convergence.The trade-off was observed experimentally and can also be shown analytically in simple settings.
The SNR is defined using the signal energy and the variance per complex baseband noise symbol.
Training uses Adam with experimentally fixed iteration counts, alternating ten receiver and ten transmitter gradient steps per iteration.The implementation uses TensorFlow and evaluates AWGN and RBF channels.
The Gaussian channel expression uses 2N-dimensional real representations of N complex baseband symbols and a matrix J built from zero and identity blocks.

A. Transmitter and receiver architectures

The evaluation combines dense neural transmitter and receiver architectures with comparisons across AWGN and RBF channels, including equalization, visualization, and training behavior.

A. Transmitter and receiver architectures: The transmitter maps one-hot messages through dense layers to 2N channel symbols, then normalizes the output.The first dense layer uses ELU activations and the output layer uses linear activations.
A. Transmitter and receiver architectures: The receiver outputs a probability distribution over M through a softmax layer; the RBF design adds a transformer before discrimination.The transformer estimates the channel response and transforms the received signal for detection.
B. Evaluation on AWGN and RBF channels: For M = 256, the comparisons include QPSK, Agrell, and model-aware training over AWGN and RBF channels.Agrell uses the E8 lattice in eight dimensions, corresponding to four complex channel uses.
B. Evaluation on AWGN and RBF channels: The model-free and model-aware methods achieve similar BLER; learning-based methods outperform QPSK and Agrell on RBF but not Agrell on AWGN.
B. Evaluation on AWGN and RBF channels: Pilot equalization matches the performance of prior-free reception with the additional transformer, while both learned methods saturate above the training SNR.The saturation is attributed to residual error from imperfect knowledge of super-imposed pilots.
B. Evaluation on AWGN and RBF channels: For RBF reception without prior equalization, t-SNE reveals message clusters after transformation but not for the received signal.The transmitter constellation indicates super-imposed pilots, which may help the transformer make messages separable.

C. Evaluation of the convergence rate

The convergence and feedback experiments assess optimization speed and robustness when training uses approximate gradients and an error-prone feedback link.

C. Evaluation of the convergence rate: Model-aware training converges faster than alternating model-free training on both AWGN and RBF channels.The difference is concentrated early in training and is consistent with using the true rather than approximate transmitter gradient.
C. Evaluation of the convergence rate: The convergence curves average BLER over 200 neural-network weight initializations and show the first 150 iterations.Shaded regions represent one standard deviation.
D. Evaluation with a noisy feedback link: Training requires a feedback link to send per-example receiver losses to the transmitter.These losses are needed for transmitter gradient estimation.
D. Evaluation with a noisy feedback link: Positive feedback-link SNR causes negligible BLER impact, and SNRfb higher than 6 dB produces no observed BLER increase.The test uses M = 256, N = 4, 10 dB AWGN, and erroneous losses modeled with additive Gaussian noise.

E. Joint source-channel coding of images

Model-free alternating training extends end-to-end communication learning to image transmission and a simplified fiber-optical channel. For MNIST over AWGN, it avoids a performance penalty relative to model-aware training, while fiber-optical results show matching SER.

MNIST image transmission: Model-free training applies to joint source-channel coding for MNIST images by adapting transmitter and receiver networks to real-valued image inputs.The receiver reconstructs an image rather than outputting message probabilities.
MNIST image transmission: The image transmitter uses convolutional layers for spatial structure and compression, then generates normalized channel symbols under a power constraint.The receiver uses transposed convolutions and sigmoid outputs for grayscale pixels in [0, 1].
AWGN experiment: 10 dB training SNR and N = 10 channel uses define the AWGN MNIST experiment, with PSNR measuring reconstruction quality.Model-free transmitter training required larger batches, attributed to increased estimator variance as transmitter outputs increased.
AWGN experiment: Model-free training incurs no performance penalty on AWGN, matching model-aware training across SNR values for MNIST reconstruction.The result indicates that alternating training handles the complex image transmitter architecture in this setting.
Fiber-optical experiment: Model-aware and model-free training achieve the same SER on the simplified fiber-optical channel.A separate network is trained for each SNR because the trained networks struggle to generalize to other SNR values.

VI. OVER-THE-AIR EXPERIMENTS

The paper demonstrates end-to-end autoencoder training directly over actual coaxial and wireless channels using an alternating algorithm. The prototype outperforms established baselines in key experimental settings, while remaining subject to training-scope limitations.

Experimental setup: The prototype is trained directly over actual wireless and coaxial cable channels, avoiding reliance on an accurate channel model during deployment-oriented training.The authors describe this as the first autoencoder-based communication system prototype trained over actual channels.
Experimental setup: The testbed uses one Ettus USRP B210 with a consumer GPU, 2.3 GHz carrier frequency, 5 MHz bandwidth, and 128 OFDM subcarriers.The wireless antennas had an unobstructed line-of-sight path and remained unmoved during transmission.
Training dynamics: Three orders of magnitude of cross-entropy loss reduction occur within a few hundred coaxial-cable training iterations.Using consumer-class hardware, a few hundred iterations take only a few minutes.
Results: The alternating algorithm outperforms QPSK over both coaxial cable and wireless channels, and outperforms Agrell over coaxial cable.On wireless, prior equalization is needed to exceed Agrell; without it, performance does not.
Discussion and outlook: The paper attributes gains over prior experimental results to jointly optimizing transmitter and receiver for the actual channel.The conclusion contrasts these gains with earlier work that obtained no experimental improvement over a well-designed baseline.
Discussion and outlook: Training requires a feedback link, and gradient-estimator variance increases with channel uses; long channel coherence times also reduce sample efficiency.The authors suggest variance reduction and artificial channel variation as possible remedies.

APPENDIX A PROOF OF THEOREM 1

The appendix establishes conditions under which smoothing the transmitter output yields a valid approximation to the missing channel gradient. It treats both noncompact and compact input-output domains.

General-domain proof: The proof assumes translation invariance and concentration of the smoothing distribution toward a Dirac distribution as its scale decreases.A domination condition controls the smoothed channel gradient with an integrable function independent of the smoothing scale.
General-domain proof: The gradient identity is derived using the log-trick, a change of variables, and translation invariance of the smoothing distribution.Regularity conditions are required to exchange integration and differentiation.
General-domain proof: Under the stated conditions, relaxing the transmitter output to a random variable approximates the channel gradient with arbitrarily good precision.The result combines the receiver-loss gradient with the smoothed transmitter-gradient expression.
Compact-domain case: For compact input and output domains, continuity of the channel gradient and uniform convergence of the smoothing distribution provide simpler sufficient conditions.Compactness supplies boundedness needed by the proof.
Compact-domain case: A continuously differentiable, compactly supported, unit-integral function can construct a smoothing distribution satisfying the required conditions.The construction scales the function around the transmitter output.

APPENDIX B GRADIENT ESTIMATION WITH SPSA

The appendix compares SPSA with the proposed gradient estimator for training transmitters with many parameters. SPSA's estimator variance grows with model size, whereas the proposed estimator remains lower and approximately size-independent in the experiment.

SPSA comparison: SPSA estimates the gradient using random perturbations with a decreasing positive sequence and a Rademacher random vector.The approach is presented as an alternative response to the missing channel-gradient problem.
SPSA comparison: SPSA fails to scale to large transmitter parameter counts because its gradient substitute does not exploit knowledge of the transmitter function.The proposed method instead estimates the gradient of the channel distribution with respect to its inputs.
Variance experiment: With 3m+1 parameters, SPSA variance increases linearly with parameter count, while the proposed estimator has much lower variance without that increase.The numerical study uses batch size S = 1000, 1000 parameter initializations, and σ = 0.1.
Variance experiment: High gradient-estimator variance causes SPSA to struggle with complex transmitter architectures containing many parameters.This conclusion follows the reported variance comparison in the regression experiment.