Source-linked AI summary
Deep Unfolding for Communications Systems: A Survey and Some New Directions
Alexios Balatsoukas-Stimming, Christoph Studer
TL;DR
Communications signal-processing problems often require accurate iterative solutions despite tight throughput and latency constraints. This survey presents deep unfolding, which combines iterative algorithms with trainable neural-network components, and reviews its use across MIMO and channel-decoding tasks, including reported gains of 0.5 dB for unfolded polar-code decoding.
Problem
Communications applications need accurate solutions to optimization-based signal-processing tasks while allowing only a small number of iterations, making parameter tuning difficult.
Method
The survey explains deep unfolding as converting iterative algorithms into fixed-depth layer-wise structures with trainable parameters learned using deep-learning methods.
Results
The survey reports deep unfolding achieving competitive or near-ML performance in MIMO detection and gains of 0.5 dB for unfolded normalized min-sum polar-code decoding.
Takeaways & Limitations
Deep unfolding provides a model-driven paradigm reviewed across MIMO detection and precoding, belief-propagation decoding, and other communications applications.
Takeaways & Limitations
Training unfolded algorithms can encounter vanishing gradients as the number of unfolded iterations increases.
Abstract
from arXiv · showhide
Deep unfolding is a method of growing popularity that fuses iterative optimization algorithms with tools from neural networks to efficiently solve a range of tasks in machine learning, signal and image processing, and communication systems. This survey summarizes the principle of deep unfolding and discusses its recent use for communication systems with focus on detection and precoding in multi-antenna (MIMO) wireless systems and belief propagation decoding of error-correcting codes. To showcase the efficacy and generality of deep unfolding, we describe a range of other tasks relevant to communication systems that can be solved using this emerging paradigm. We conclude the survey by outlining a list of open research problems and future research directions.
I. INTRODUCTION
Communications signal-processing tasks often require iterative optimization under stringent throughput and latency constraints, motivating deep unfolding as a model-driven alternative to black-box neural networks. The survey explains its principle, applications, and research directions.
- I. INTRODUCTION: High-throughput, low-latency communications applications may permit only ten or fewer algorithm iterations, making parameter tuning important for accurate solutions.The introduction highlights step-size selection as an example of the tuning required when iteration counts are small.
- I. INTRODUCTION: Black-box neural methods can offer promising performance but have high computational and memory demands and virtually no performance guarantees.
- I. INTRODUCTION: Deep unfolding unfolds a fixed number of iterative-algorithm steps into neural-network-like layers and learns trainable parameters using deep-learning techniques.The described training tools include suitable loss functions, stochastic gradient descent, and back-propagation.
- I. INTRODUCTION: The survey focuses on deep unfolding for MIMO detection and precoding, belief-propagation decoding, other communications tasks, and open research directions.
II. DEEP UNFOLDING FOR MIMO SYSTEMS
The survey examines deep unfolding for signal-processing tasks in MIMO wireless systems, emphasizing data detection and precoding. It summarizes recent MIMO detection results and the discussed precoding application in Table I.
- II. DEEP UNFOLDING FOR MIMO SYSTEMS: Deep unfolding is applied to signal-processing tasks in multiple-input multiple-output wireless systems.
- II. DEEP UNFOLDING FOR MIMO SYSTEMS: The MIMO applications discussed include recent results on data detection.
- II. DEEP UNFOLDING FOR MIMO SYSTEMS: The section also covers MIMO precoding, with the detection and precoding methods summarized in Table I.
A. MIMO Data Detection
Deep unfolding applies trainable, layer-wise versions of iterative MIMO detection algorithms to approximate maximum-likelihood detection at reduced computational cost. Across detector variants, learned parameters achieve near-ML or improved performance under several channel and constellation settings.
- Problem: MIMO data detection estimates transmitted symbols from received signals, channel information, and noise statistics, while maximum-likelihood detection is NP-hard.Approximate methods are therefore used in practice, including real-valued formulations of the complex channel model.
- Projected-gradient unfolding: Projected-gradient detectors in and unfold iterative detection updates and enrich them with additional dimensions, trainable parameters, and learned nonlinearities.The unfolded networks use layer-specific weights, biases, soft-sign or sigmoid operators, and auxiliary dimensions.
- Higher-order constellations: Close-to-ML performance is achieved for a five-level constellation over a fixed 8×8 MIMO channel using shifted sigmoid detectors and ML-detectable training samples.The method trains two detectors with different initialization strategies and retains the better output.
- OAMP unfolding: Data-driven tuning of γt and θt produces significant improvements over standard OAMP for both Rayleigh and correlated channels.The unfolded OAMP detector introduces only the trainable scalars γt and θt per iteration.
- Massive overloaded MIMO: Trained projected-gradient detection achieves similar performance to other detectors at significantly lower complexity and also performs well in non-overloaded MIMO systems.The method targets massive overloaded channels with N ≫ M and uses incremental training to avoid vanishing gradients.
- Recent directions: Conjugate-gradient and projected-gradient deep-unfolding detectors achieve near-ML performance at low complexity.These results extend the reported MIMO detection approaches beyond the earlier projected-gradient and OAMP variants.
B. Multi-User (MU) MIMO Precoding
Deep unfolding applies trainable projected-gradient iterations to massive MU-MIMO precoding with 1-bit transmitter quantization. Learning suitable parameters reduces the iterations needed for a given error-rate performance.
- MU-MIMO precoding multiplies the transmit symbol vector by a precoding matrix to maximize a performance metric such as receiver SNR.
- In 1-bit transmitter quantization, each element of the transmit vector is constrained to {-υ, +υ} under a transmit-power constraint.
- The unfolded precoding algorithm uses trainable scalars τt and ρt across T projected-gradient iterations.
- A factor-of-two reduction in iterations achieves the same error-rate performance across a range of channel models.
III. DEEP UNFOLDING FOR BELIEF-PROPAGATION-BASED CHANNEL DECODING
Deep unfolding augments belief-propagation and min-sum decoding with trainable message weights, normalization parameters, and loss functions. Across BCH, polar, LDPC, and finite-alphabet decoding, the surveyed methods report improved performance or reduced complexity relative to conventional decoders.
- Belief propagation: Belief propagation decodes error-correcting codes by iteratively passing messages between variable and check nodes in a Tanner graph.
- Unfolded BP: A binary cross-entropy loss using outputs from all iterations helps train unfolded BP decoders and avoid vanishing gradients.
- Unfolded BP: Unfolded BP with learned weights significantly outperforms traditional BP decoders on a variety of BCH codes.
- Unfolded min-sum decoding: Per-message learned normalization parameters improve polar-code decoding by approximately 0.5 dB over standard normalized min-sum decoding.
- Complexity reduction: Using a single shared weight has negligible error-rate impact while significantly reducing learning and decoding complexity.
- Parameterization: Learned additional parameters improve unfolded min-sum decoding over standard min-sum, belief propagation, and earlier unfolded min-sum methods.
- Finite-alphabet decoding: Learning finite-alphabet LDPC decoders yields gains of up to 0.25 dB for a (1296, 972) QC-LDPC code with 3 quantization bits.
IV. DEEP UNFOLDING FOR OTHER COMMUNICATIONS APPLICATIONS
Deep unfolding has been applied beyond MIMO detection, precoding, and BP decoding to other communication-system tasks. These include Turbo and successive-cancellation decoding, channel-dependent Viterbi processing, nonlinear signal processing, optical and full-duplex communications, and sparse channel estimation.
- Unfolding has been studied for Turbo decoding, successive-cancellation decoding of polar codes, and replacing channel-dependent Viterbi components with a deep neural network.
- For nonlinear signal processing, unfolding can represent nonlinear equations rather than iterations, including models used in optical and full-duplex communications.
- ISTA unfolding addresses sparse linear inverse problems such as sparse channel estimation in communication systems.
V. FUTURE RESEARCH DIRECTIONS
Deep unfolding remains relatively novel, with many open research questions despite wide applicability in communication systems. The survey highlights proximal algorithms as a promising class for future unfolding applications.
- Many open research questions remain even though deep unfolding has already been applied widely in communication systems.
- Proximal algorithms solve a wide range of communication-system optimization problems and are generally well-suited for unfolding.
A. Unfolded Structures with Acceleration Methods
Optimization techniques such as preconditioning, momentum, Onsager terms, restart, and adaptive step sizes can be incorporated into unfolded architectures. These methods are especially relevant when only a severely limited number of iterations is available.
- Preconditioning, momentum, Onsager terms, restart, and adaptive step-size rules can accelerate unfolded optimization algorithms.
- Acceleration methods are particularly important when unfolded applications impose severe iteration constraints.
B. Loss Functions
Application-tailored loss functions can improve training convergence and may support real-time or online training. Although MSE is convenient for MIMO detection and precoding, it may not accurately represent system error-rate performance.
- Application-tailored cost functions can improve convergence for the same computational effort and may enable real-time or online training.
- Most surveyed MIMO detection and precoding methods use standard MSE because it permits closed-form solutions or accurate iterative approximations.
- MSE may poorly proxy error-rate performance in multi-user MIMO, where the user with the largest MSE can dominate system error rate.
- Alternative objectives include maximum MSE across users or across users and channel realizations in the training dataset.
C. Training
Training unfolded algorithms is generally straightforward with few iterations but becomes more difficult as the iteration count grows. Vanishing gradients and memory demands motivate multi-loss, incremental, windowed, and initialization-based strategies.
- Training becomes increasingly difficult with more unfolded iterations because vanishing gradients hinder parameter learning in early iterations.
- Multi-loss functions, incremental training, and known good initial values can address training difficulties caused by vanishing gradients.
- Windowed training can reduce memory requirements by training unfolded iterations over a moving fixed-size window.
- Memory can become limiting for high-dimensional inputs such as massive MIMO and many iterations because intermediate mini-batch outputs are stored for back-propagation.
D. Hardware Implementation
Unfolded algorithms resemble known algorithms with existing efficient hardware architectures, making them attractive for implementation. However, the impact of additional trainable parameters on hardware complexity and throughput remains unclear.
- Unfolded learned algorithms are attractive for hardware implementation because they resemble known algorithms with efficient existing architectures.
- The literature typically does not assess how additional trainable parameters affect hardware implementation complexity and achieved throughput.
- Efficient hardware implementations of training are needed for applications requiring online learning.