PaperlayerView exact revision on arXiv

Source-linked AI summary

End-to-end Deep Learning of Optical Fiber Communications

Boris Karanov, Mathieu Chagnon, Félix Thouin, Tobias A. Eriksson, Henning Bülow, Domaniç Lavery, Polina Bayvel, Laurent Schmalen

arXiv:1804.04097v3cs.ITstat.ML

TL;DR

Conventional block-wise optimization can be sub-optimal for optical systems whose optimal receivers are unknown or too complex. This paper jointly learns the transmitter, channel model, and receiver as an end-to-end deep neural network for IM/DD communication, achieving below-threshold experimental transmission beyond 40 km.

  • Problem

    Separate optimization of communication blocks can be sub-optimal when optimum receivers or blocks are unknown or too complex to implement.

  • Method

    The paper models the complete IM/DD transmitter, nonlinear channel, and receiver as one deep feed-forward neural network trained end to end.

  • Results

    42 Gb/s information rates were experimentally achieved beyond 40 km with BERs below the 6.7% HD-FEC threshold, outperforming PAM2/PAM4 with conventional receiver equalization over tested distances.

  • Takeaways & Limitations

    End-to-end deep learning provides a demonstrated route to optical-fiber transceivers that jointly account for transmitter, channel, and receiver behavior.

Abstract

from arXiv · show

In this paper, we implement an optical fiber communication system as an end-to-end deep neural network, including the complete chain of transmitter, channel model, and receiver. This approach enables the optimization of the transceiver in a single end-to-end process. We illustrate the benefits of this method by applying it to intensity modulation/direct detection (IM/DD) systems and show that we can achieve bit error rates below the 6.7\% hard-decision forward error correction (HD-FEC) threshold. We model all componentry of the transmitter and receiver, as well as the fiber channel, and apply deep learning to find transmitter and receiver configurations minimizing the symbol error rate. We propose and verify in simulations a training method that yields robust and flexible transceivers that allow---without reconfiguration---reliable transmission over a large range of link dispersions. The results from end-to-end deep learning are successfully verified for the first time in an experiment. In particular, we achieve information rates of 42\,Gb/s below the HD-FEC threshold at distances beyond 40\,km. We find that our results outperform conventional IM/DD solutions based on 2 and 4 level pulse amplitude modulation (PAM2/PAM4) with feedforward equalization (FFE) at the receiver. Our study is the first step towards end-to-end deep learning-based optimization of optical fiber communication systems.

I. INTRODUCTION

Conventional optical communication systems optimize separate processing blocks, which can be sub-optimal when optimal receivers or blocks are unknown or too complex. This paper instead optimizes transmitter, channel, and receiver jointly as one deep neural network, focusing on supervised offline training for IM/DD systems.

  • Motivation: Block-based optimization can be sub-optimal when optimum receivers or processing blocks are unknown or too complex to implement.Conventional systems separately analyze, optimize, and control coding, modulation, and equalization blocks.
  • Proposed approach: End-to-end deep learning jointly optimizes the transmitter, communication channel, and receiver in a single process.The communication system is implemented as a single deep neural network rather than a conventional modular structure.
  • Scope: The paper introduces end-to-end deep learning for optical-fiber transceiver design and applies it to nonlinear IM/DD channels.The modeled system includes transmitter and receiver componentry, fiber dispersion, photodetection, amplifier noise, and DAC/ADC quantization noise.
  • Reported outcomes: The target is reliable communication below FEC thresholds, with experimental information rates of 42 Gb/s beyond 40 km.The paper also reports a simulation-based training method for flexible transceivers operating across a range of distances.
  • Training scope: Training is supervised and performed offline so the deployed transceiver need not implement training in the final product.The paper limits its scope to offline supervised training and identifies online or unsupervised training as future work.
  • Neural-network foundations: The neural-network framework uses fully connected feed-forward layers, nonlinear activations, supervised loss minimization, backpropagation, and Adam optimization.ReLU activations support nonlinear approximation, while Adam dynamically adapts the learning rate during optimization.

III. PROPOSED END-TO-END COMMUNICATION SYSTEM

The proposed system represents the complete optical-fiber transmission chain—including transmitter, channel, and receiver—as one end-to-end artificial neural network. Although demonstrated for IM/DD, the framework is presented as extensible to more complex channel models.

  • System architecture: The complete fiber-optic transmission chain is implemented as a single end-to-end feed-forward neural network.The network includes the transmitter, communication channel, and receiver, with the full chain depicted in Fig. 1.
  • Scope: The demonstration focuses on IM/DD, while the general method is not restricted to that scheme.The authors state that the approach can be extended to other, eventually more complex, channel models.

A. Transmitter Section

The transmitter uses block-based message encoding implemented by neural-network layers. Messages are represented as one-hot vectors and mapped to transmit-sample vectors before waveform generation and hardware modeling.

  • Block-based transmission: Block-based transmission encodes each independent message into a vector of n transmit samples.The approach is computationally simple and supports massive parallel processing of individual blocks.
  • Message encoding: Each message is represented by a one-hot vector of size M before entering the transmitter network.One-hot encoding avoids imposing an undesired ordering on categorical messages and facilitates symbol-error-rate minimization.
  • Hardware constraints: The transmitter waveform is constrained for modulator operation by accounting for DAC quantization noise.The margin term ε = σq/2 keeps the signal within the MZM limits after quantization noise is added.

B. Communication Channel

The IM/DD channel model captures bandwidth limits, dispersion, nonlinear modulation and photodetection, quantization, and additive noise within the end-to-end neural network. Fiber dispersion is applied in the frequency domain, while linear and nonlinear hardware stages are represented by corresponding network operations.

  • Channel properties: IM/DD intersymbol interference arises from fiber dispersion, while square-law photodetection makes the channel nonlinear and gives it memory.These joint effects form the main channel limitation described for IM/DD systems.
  • Channel model: The modeled channel includes LPF, DAC, ADC, MZM, photodetection, amplification noise, and optical-fiber transmission.The channel is treated as part of the end-to-end feed-forward neural network.
  • Photodetection: After fiber propagation, photodetection produces r(t) = |u(t)|^2 + n_Rec(t).The received signal combines the squared propagated optical-field waveform with additive Gaussian receiver noise.
  • Fiber dispersion: Chromatic dispersion is modeled by a frequency-domain transfer function derived from the fiber propagation equation.The implementation applies dispersion to a zero-padded signal using FFT and IFFT operations within the neural-network chain.
  • Neural-network implementation: LPF and dispersion stages are modeled as linear matrix operations, whereas MZM and photodiode stages use nonlinear functions.The MZM uses an element-wise sine transfer function, and DAC/ADC quantization noise is modeled as additive uniformly distributed noise.

C. Receiver Section

The receiver reverses the transmitter-side architecture to map processed samples to a probability distribution over messages, from which symbol decisions and errors are determined. BER is evaluated using Gray-coded bits, while blockwise processing limits dispersion compensation to each receiver block.

  • Receiver architecture: The receiver ANN uses reverse-order layers, ending with softmax outputs that form a probability vector over the encoded messages.A symbol error occurs when the maximum-probability message differs from the transmitted message.
  • Error decision: A block error is counted when argmax(y) differs from the transmitted message index, defining the block error rate.The indicator function contributes one for an incorrect decision and zero otherwise.
  • Bit-error evaluation: BER uses Gray coding, but this mapping is sub-optimal because the network minimizes block error rate rather than bit errors.A symbol error can therefore produce more than one wrong bit.
  • Blockwise limitation: The receiver compensates chromatic dispersion only within each block, while dispersion from neighboring blocks is treated as extra noise.Larger jointly processed blocks could improve dispersion resilience but would increase computation complexity.

D. Training

Training treats the transceiver as an autoencoder whose softmax output should reproduce the one-hot input message. The study reports convergence, while noting that hyper-parameter optimization was outside its scope.

  • Training objective: The training objective is to make the final softmax output match the one-hot input, thereby minimizing end-to-end block error rate.Optimization uses Adam on randomly chosen messages and neighboring transmit-block messages.
  • Optimization procedure: Training uses 25 × 10^6 messages, mini-batches of 250, and 100,000 optimization iterations.Validation is performed every 5,000 iterations using a validation set of 15 × 10^6 messages.
  • Initialization and validation: Weights are initialized from a truncated normal distribution with standard deviation 0.1, while biases are initialized to zero.The trained model is saved and loaded separately for testing after validation convergence.
  • Scope: The reported networks use relatively small designs because hyper-parameter optimization was beyond the investigation’s scope.The authors state that better performance and complexity trade-offs may be possible with well-designed hyper-parameters.

IV. SYSTEM PERFORMANCE

Simulations evaluate end-to-end systems across distance, training strategy, and information rate, with BER compared against the 6.7% HD-FEC threshold. Fixed-distance training gives strong performance near its design point, while distance-distributed training broadens the reliable operating range at a performance cost.

  • Simulation setup: 42 Gb/s systems use 6-bit messages encoded into 48 samples at 336 GSa/s, with a 32 GHz low-pass filter in simulation.The sampling rate corresponds to an 84 GSa/s experimental DAC with an assumed oversampling factor of four.
  • Distributed-distance training: Training over normally distributed distances produces more generalized parameters that tolerate dispersion variation without reconfiguration.The reported example uses a mean distance of 40 km and compares standard deviations of 4 and 10 km.
  • Distributed-distance results: 27.75 km of transmission range, from 24 km to 51.75 km, remains below the HD-FEC threshold after distance-distributed training.The generalized system has higher minimum BER than fixed-distance training, establishing a robustness–performance trade-off.
  • Information-rate results: At 84 Gb/s, BER below the HD-FEC threshold is achieved only up to 20 km, while 42 Gb/s and 56 Gb/s are transmitted reliably at 30 km.These rates are obtained by varying message size and encoded samples while keeping the simulation sampling rate at 336 GSa/s.

V. EXPERIMENTAL VALIDATION

The end-to-end deep-learning IM/DD system was validated experimentally at multiple fiber distances and dispersion settings, with receiver retraining used to account for experimental distortions. It outperformed PAM2/PAM4 comparisons at 20–40 km but encountered noise and block-processing limits at longer links.

  • 42 Gb/s transmission was experimentally evaluated at nominal distances of 20, 40, 60, and 80 km with dispersion sweeps.The training used SNR values corresponding to the measured PAM2 reference system at each distance.
  • Receiver retraining was required because the simulation-trained ANN did not fully compensate distortions introduced by the experimental setup.The transmitter was kept fixed, and retraining was performed separately for each measured distance.
  • 3.73 × 10^-6 was the lowest measured BER at 21.18 km, while the end-to-end system achieved 1.05 × 10^-3 at 40 km.At 20 km, the proposed system also significantly outperformed the experimental PAM2 BER of 7.77 × 10^-4.
  • At 40 km, the proposed system achieved BERs below the HD-FEC threshold while 42 Gbaud PAM2 and 21 Gbaud PAM4 did not.The effective swept ranges were 17.65–22.35 km and 37.65–42.35 km for nominal 20 km and 40 km links.
  • At 60 and 80 km, BERs below the HD-FEC threshold were not achieved without amplification, and 80 km performance was restricted by thermal noise and accumulated dispersion.At 80 km, dispersion extended across multiple blocks and could not be compensated by block-by-block processing.
  • Adding an EDFA improved BER to 3.8 × 10^-3 at 60 km and 2.8 × 10^-2 at 80 km.The EDFA increased received power from -13/-17 dBm to -7 dBm at 60/80 km.

A. Transmitted Signal Characteristics

The learned transmitter maps messages to optimized waveform blocks whose amplitudes are constrained by the modulator and whose waveform representations remain separable after learning. Filtering confines the signal to the hardware bandwidth while block transmission creates symbol-rate harmonics.

  • Each 6-bit symbol is represented by a 48-sample waveform transmitted at 7 GSym/s, corresponding to a symbol duration of approximately 143 ps.The transmitter ANN maps each message to one optimized waveform block.
  • The waveform amplitude is limited to the Mach–Zehnder modulator’s linear operating region by the transmitter clipping layer.Filtering causes only a small departure from the stated amplitude range.
  • The transmitter ANN behaves as a lookup table mapping each of 64 messages to one optimized block of 48 amplitude samples.Figure 9 displays the amplitude levels for all 64 possible transmitter outputs.
  • The 64 learned waveforms are well separated in a two-dimensional t-SNE representation and can therefore be discriminated sufficiently.The transmitter outputs predominantly use the extremal levels, although intermediate values also occur.
  • Low-pass filtering confines the transmitter spectrum within 32 GHz and introduces strong harmonics at multiples of the symbol rate.Transmitter and receiver filters keep the signal within finite hardware bandwidth.
  • After DAC, optical modulation, fiber propagation, and direct detection, distorted received blocks are processed by the receiver ANN for equalization.The receiver operates on the transmitted blocks after the complete IM/DD channel chain.

B. Comparison with Receiver-Only and Transmitter-Only ANN-Processing

The study compares jointly optimized end-to-end transceivers with PAM2/PAM4 and separately structured ANN alternatives. End-to-end optimization substantially improves simulated error rates, while a transmitter ANN with a linear receiver offers similar performance and somewhat greater distance robustness.

  • The comparison includes PAM2/PAM4 with ANN equalization, an ANN transmitter with linear classification, separately trained transmitter and receiver ANNs, and end-to-end learning.All systems are compared at 42 Gb/s with 6 bits of information per symbol block.
  • At 20 km, the end-to-end system achieves symbol error rate below 10^-6, versus 5.5 × 10^-4 for PAM2 and 2.9 × 10^-3 for PAM4.The end-to-end design significantly outperforms both PAM counterparts at the 20 km training distance.
  • The end-to-end ANN design significantly outperforms PAM2 and PAM4 systems for training distances of 20 km and 40 km.Figure 12 reports symbol error rate as a function of transmission distance for the compared architectures.
  • The linear receiver uses one fully connected softmax layer to map 48 received samples to a probability vector over 64 message classes.The decoded message is the class corresponding to the largest softmax output.
  • The transmitter-ANN and linear-receiver system achieves symbol error rate close to the deep end-to-end design when trained at 20 km.Its slightly better robustness to distance variations is attributed to the absence of a receiver deep ANN optimized specifically at the nominal distance.

3) ANN-based transmitter & ANN-based receiver, separately trained:

The benchmark separately trains transmitter and receiver ANNs by fixing a PAM2-optimized receiver and optimizing the transmitter to match it. This separately trained arrangement cannot outperform the jointly trained end-to-end system, while experimental receiver retraining addresses model-to-hardware discrepancies.

  • Separately trained benchmark: The benchmark fixes receiver ANN parameters previously optimized for PAM2 transmission and trains only the transmitter ANN to match that receiver.Training remains end-to-end, but optimization is not jointly applied to both ANNs.
  • Performance comparison: The separately trained transmitter-and-receiver ANNs cannot outperform the proposed jointly trained end-to-end system.
  • Experimental adaptation: Experimental receiver retraining was used to account for discrepancies between the simulated channel model and the real experimental setup.Fine-tuning initializes receiver parameters from simulation before retraining with labeled experimental data.
  • Experimental methodology: Experimental ANN evaluation follows safeguards against learning representations of fixed sequences that could bias BER estimates downward.The work uses newly generated random input messages during offline training and a long random sequence rather than a PRBS for validation.
  • Experimental outcome: The study experimentally demonstrates 42 Gb/s transmission beyond 40 km with BER below the 6.7% HD-FEC threshold.The conclusion also reports that the proposed system outperforms PAM2/PAM4 with conventional receiver equalization across transmission distances.
Loading 1804.04097v3…