PaperlayerView exact revision on arXiv

Source-linked AI summary

Iterative Algorithm Induced Deep-Unfolding Neural Networks: Precoding Design for Multiuser MIMO Systems

Qiyu Hu, Yunlong Cai, Qingjiang Shi, Kaidi Xu, Guanding Yu, Zhi Ding

arXiv:2006.08099v1cs.ITcs.LGeess.SP

TL;DR

Iterative optimization algorithms for MU-MIMO precoding offer strong performance but are computationally demanding for real-time systems. The paper develops a matrix-form IAIDNN framework and applies it by unfolding WMMSE with trainable parameters and a generalized chain rule. The resulting IAIDNN achieves iterative-WMMSE performance with reduced computational complexity.

  • Problem

    Iterative MU-MIMO precoding algorithms provide excellent performance but require considerable computational complexity, limiting their practical application in real-time systems.

  • Method

    The paper develops a matrix-form IAIDNN framework and an efficient WMMSE-based IAIDNN with trainable parameters replacing high-complexity forward-propagation operations.

  • Results

    The proposed IAIDNN efficiently achieves the performance of iterative WMMSE with reduced computational complexity.

  • Takeaways & Limitations

    Deep unfolding can provide an efficient MU-MIMO precoding scheme while retaining the performance of the iterative WMMSE algorithm.

Abstract

from arXiv · show

Optimization theory assisted algorithms have received great attention for precoding design in multiuser multiple-input multiple-output (MU-MIMO) systems. Although the resultant optimization algorithms are able to provide excellent performance, they generally require considerable computational complexity, which gets in the way of their practical application in real-time systems. In this work, in order to address this issue, we first propose a framework for deep-unfolding, where a general form of iterative algorithm induced deep-unfolding neural network (IAIDNN) is developed in matrix form to better solve the problems in communication systems. Then, we implement the proposed deepunfolding framework to solve the sum-rate maximization problem for precoding design in MU-MIMO systems. An efficient IAIDNN based on the structure of the classic weighted minimum mean-square error (WMMSE) iterative algorithm is developed. Specifically, the iterative WMMSE algorithm is unfolded into a layer-wise structure, where a number of trainable parameters are introduced to replace the highcomplexity operations in the forward propagation. To train the network, a generalized chain rule of the IAIDNN is proposed to depict the recurrence relation of gradients between two adjacent layers in the back propagation. Moreover, we discuss the computational complexity and generalization ability of the proposed scheme. Simulation results show that the proposed IAIDNN efficiently achieves the performance of the iterative WMMSE algorithm with reduced computational complexity.

I. INTRODUCTION

The paper addresses the high complexity and limited practical suitability of iterative MU-MIMO precoding algorithms by developing a matrix-form deep-unfolding framework. It instantiates this framework as an IAIDNN based on WMMSE, achieving WMMSE-level performance with reduced complexity.

  • Motivation: Iterative MU-MIMO precoding algorithms approach theoretical-bound performance but incur high complexity from large matrix inversions and many iterations.These costs are especially significant in massive MU-MIMO systems and hinder real-time applications.
  • Motivation: Black-box neural networks have poor interpretability and generalization, lack performance guarantees, and require many training samples.The resulting training process can involve long training times.
  • Contributions: The paper proposes a general matrix-form iterative algorithm induced deep-unfolding neural network framework for communication problems.The framework unfolds an iterative algorithm into a layer-wise structure and introduces trainable parameters in forward propagation.
  • Contributions: A generalized chain rule describes gradient recurrence between adjacent layers and calculates gradients for trainable parameters during back propagation.The paper states that the conventional DNN chain rule is a special case of the proposed generalized chain rule.
  • Application: For MU-MIMO sum-rate maximization, the proposed IAIDNN unfolds the classic WMMSE algorithm into layers and uses trainable parameters to reduce high-complexity operations.The design approximates matrix inversion using a first-order Taylor expansion and updates parameters with stochastic gradient descent.
  • Results: Simulation results show that IAIDNN achieves iterative-WMMSE performance with reduced computational complexity and outperforms conventional precoding algorithms and a black-box CNN.The reported benefit becomes more significant in massive MU-MIMO systems.

II. PROPOSED DEEP-UNFOLDING BASED FRAMEWORK

The paper develops a matrix-form framework that converts general iterative optimization algorithms into trainable deep-unfolding neural networks. The framework introduces layer parameters to reduce iterative complexity and provides a generalized chain rule for training.

  • A general matrix-form IAIDNN framework unfolds an iterative algorithm into a layer-wise neural network.Each layer represents an iteration mapping with trainable parameters.
  • The generalized chain rule is developed in matrix form to calculate gradients between adjacent network layers.It supports back propagation for the trainable parameters of the unfolded network.

B. Forward Propagation

The forward-propagation network replaces iterative mappings with trainable layer transformations while preserving the objective function as the training loss. Its matrix-form gradient recurrence generalizes the conventional neural-network chain rule.

  • Forward Propagation: The unfolded network maps X_l−1 to X_l through layer-specific functions F_l with trainable parameters θ_l and input Z.The final output X_L is substituted into the objective function to obtain the network result.
  • Forward Propagation: The original objective function f(X; Z) serves as the neural-network loss after evaluating the final output X_L.This connects the optimization problem directly to network training.
  • Back Propagation: The generalized chain rule recursively propagates gradients from the final layer through adjacent layers to each trainable parameter.The process starts with the gradient of X_L, obtains gradients for each layer, and then computes parameter gradients.
  • Back Propagation: The proposed matrix-form chain rule contains the conventional scalar DNN chain rule as a special case.The comparison identifies particular matrix choices that reduce the generalized rule to the existing formulation.
  • Back Propagation: A quadratic network structure with element-wise nonlinear operations is derived as a concrete application of the generalized gradient recurrence.The resulting corollary is used to design the IAIDNN in the subsequent section.

III. ITERATIVE WMMSE PRECODING DESIGN ALGORITHM

The classic WMMSE algorithm solves MU-MIMO precoding by maximizing weighted system sum-rate under a transmit-power constraint. Its block-coordinate updates are transformed into an unconstrained MMSE problem and an iterative mapping suitable for unfolding.

  • Problem Formulation: The MU-MIMO design problem maximizes system sum-rate subject to a total transmit-power constraint.User priorities are represented by weights ω_k, and the base station uses precoding matrices V_k.
  • WMMSE Reformulation: The WMMSE formulation introduces auxiliary variables and is equivalent to the original sum-rate maximization problem in the precoding variables.This yields a more tractable unconstrained MMSE problem after incorporating a scaling operation.
  • Iterative Algorithm: The block-coordinate descent WMMSE algorithm converges to a stationary point of the original problem.The algorithm alternates updates of the auxiliary variables and precoders, followed by power-constraint scaling.
  • Framework Identification: The optimization variables correspond to {W_k, U_k, V_k}, while the problem parameters include channels, user weights, noise powers, and the transmit-power budget.This identification connects the WMMSE problem to the general framework.
  • Iterative Algorithm: The WMMSE iterations are expressed as mappings F_t, G_t, and J_t, matching the general iterative-algorithm form used by the deep-unfolding framework.The iterative process is depicted in the WMMSE flowchart.

IV. PROPOSED IAIDNN FOR PRECODING DESIGN

The proposed IAIDNN unfolds the classic WMMSE precoding algorithm and replaces expensive matrix inversions with lower-complexity trainable structures. It outputs selected WMMSE variables, retains the final precoder update, and uses a loss based on expected channel realizations.

  • IAIDNN Architecture: The IAIDNN is designed by unfolding the classic iterative WMMSE precoding algorithm into a neural-network architecture.The architecture is illustrated in Fig. 2, with circles denoting nonlinear operations and squares denoting trainable parameters.
  • Matrix-Inversion Approximation: High-complexity matrix inversion is approximated using element-wise reciprocal operations, trainable matrix multiplications, and affine correction terms.The approximation combines A+X with AY+Z, where Y and Z are trainable.
  • Matrix-Inversion Approximation: The approximation applies A+X+AY+Z to replace the matrix inverse A−1 in the unfolded layers.Trainable parameter sets and offsets are introduced for the inversion-related variables.
  • IAIDNN Architecture: Because U_k and W_k are lower-dimensional than V_k, the IAIDNN treats them as network outputs and computes the final V_k iteratively in the last layer.This preserves the WMMSE precoder update at the network output.
  • Training Objective: The network uses the expected channel matrix in its objective and treats the resulting objective function as the training loss.Each layer also normalizes V_k to avoid gradient explosion and satisfy the power constraint.

B. Generalized Chain Rule and Back Propagation

The IAIDNN is trained by propagating gradients through its layered iterative structure. A generalized chain rule establishes recurrence relations between adjacent layers, enabling parameter updates by gradient descent.

  • Parameter Updates: The IAIDNN parameters are updated using gradient descent after gradients are computed from the objective function.The gradients of trainable parameters are obtained through the proposed recurrence and then used for updates.
  • Generalized Chain Rule: The generalized chain rule derives gradient recurrences between adjacent IAIDNN layers.Gradients with respect to layer variables are propagated backward from layer l+1 to layer l.
  • Training Setup: Training uses decreasing step sizes satisfying m(σm)^2 < ∞, with σm = m^−α for 0 < α < 1 as an example.Trainable parameters are randomly initialized, while V0_k uses a zero-forcing precoder initialization.
  • Training Procedure: The training and testing stages are distinguished, and detailed IAIDNN training procedures are provided in Algorithm 2.The training-stage and testing-stage process is summarized in Fig. 3.

V. COMPUTATIONAL COMPLEXITY AND GENERALIZATION ABILITY

The paper compares the proposed IAIDNN with a black-box CNN and specifies their training procedures. The IAIDNN retains algorithmic structure, whereas the benchmark CNN learns mappings through convolutional and fully connected layers.

  • Benchmark CNN: The black-box CNN is introduced as a benchmark for comparison with the proposed IAIDNN.Its architecture and parameter dimension are analyzed alongside those of the IAIDNN and iterative WMMSE.
  • IAIDNN Training: The IAIDNN training algorithm uses forward propagation, gradient back propagation, mini-batch parameter updates, and convergence-based termination.The procedure initializes samples and variables, computes the final-layer output, averages batch gradients, and stops when validation loss converges or m reaches Imax.
  • CNN Architecture: The benchmark CNN processes channel inputs through repeated convolution, batch-normalization, and nonlinear layers before a fully connected layer.Flattening and dropout are applied in the fully connected stage, with leaky ReLU as the nonlinear function.
  • CNN Outputs: The CNN outputs auxiliary variables Uk and Wk rather than Vk because the former have lower dimension and are easier to learn.The auxiliary variables are subsequently inserted into the iterative expression to calculate Vk.
  • Training Strategy: The black-box CNN combines supervised pretraining using WMMSE-generated labels with subsequent unsupervised learning based on the system loss function.Unsupervised training terminates when the validation loss converges.

B. Parameter Dimension and Computational Complexity

The IAIDNN reduces inference complexity relative to iterative WMMSE by using fewer layers and avoiding matrix inversion, while closed-form gradients also shorten training time compared with the black-box CNN.

  • Parameter Dimension: The parameter dimensions of the IAIDNN, black-box CNN, and iterative WMMSE are analyzed using their respective layer and matrix structures.For the CNN, convolutional and fully connected layers contribute separately to the parameter dimension.
  • Inference Complexity: The IAIDNN has lower complexity than iterative WMMSE because it uses far fewer layers, with La << Lw.The iterative algorithm also requires matrix inversion, whereas the IAIDNN avoids that operation in inference.
  • Training Complexity: The IAIDNN uses closed-form gradients to achieve much shorter training time than the conventional black-box CNN.The training-time advantage is especially pronounced during unsupervised training.

C. Analysis of Generalization Ability

Simulation results examine transferability, sum-rate behavior, convergence, robustness, and data efficiency across system configurations. The IAIDNN generally approximates iterative WMMSE more closely than the black-box CNN, with performance depending on system load and network depth.

  • Straightforward Extension: The trained IAIDNN can transfer to systems with fewer users by zero-padding the missing user channels, without retraining a new network.The described example transfers from (Nt0, Nr0, K0) = (32, 2, 10) to (Nt1, Nr1, K1) = (16, 2, 5) using zero-padded inputs.
  • Generalization Ability: The IAIDNN is also applicable to robust precoding with CSI errors and to multicell systems by incorporating CSI-error statistics and adjusting the network.Transmit power and noise can likewise be treated as network inputs rather than fixed quantities.
  • Sum-Rate Performance: As K × Nr approaches Nt, IAIDNN performance degrades while the conventional black-box CNN deteriorates severely.The paper attributes the CNN difficulty to Vk elements becoming close to 0 or 1, making satisfactory learning harder.
  • Convergence: The proposed learning-rate scheme balances convergence speed and sum-rate performance, while smaller batch sizes can increase achieved sum-rate but reduce stability.Larger learning rates converge faster, whereas smaller learning rates achieve better sum-rate performance.
  • Sum-Rate Performance: 99.82% of iterative-WMMSE sum-rate is achieved by IAIDNN for K = 10, compared with 92.83% for the black-box CNN.The performance gaps among the schemes increase with the number of users, and the black-box CNN has the largest variance.
  • Training Data: The IAIDNN needs fewer training samples than the black-box CNN because it uses the structure of the classic iterative WMMSE algorithm.This difference is relevant when training data and channel-estimation implementation are costly.
  • Robustness: With increasing CSI error, performance degrades, but the proposed IAIDNN provides the best performance among the compared schemes.The reported ordering places iterative WMMSE second and the black-box network behind it, while describing IAIDNN as more robust than iterative WMMSE.

C. Complexity Comparison

The IAIDNN reduces computational time relative to iterative WMMSE and black-box CNN approaches, with the advantage becoming more pronounced as system dimensions increase. Its closed-form gradient computation also shortens training and accelerates convergence.

  • Complexity trends: CPU time increases with transmit antennas N_t and users K in both training and testing stages, while training grows more quickly.The training stage involves more matrix multiplication and inversion operations.
  • Training efficiency: The proposed IAIDNN has shorter training time and converges faster than the black-box CNN.The paper attributes this to closed-form gradient computation that avoids inefficient gradient calculation through TensorFlow for the complicated loss function.
  • Training complexity: The training-time CPU gap between IAIDNN and black-box CNN becomes larger as N_t and K increase.This comparison concerns the training stage across varying system dimensions.
  • Testing complexity: The IAIDNN requires shorter testing-stage CPU time than the iterative WMMSE algorithm, especially for N_t = 128 or 256.The reported advantage is more significant in large-scale MU-MIMO scenarios.
  • Practical implication: The shorter testing time makes iterative-algorithm-based precoding more applicable to practical engineering systems.The stated benefit is particularly relevant to large-scale MU-MIMO settings.

D. Generalization Ability

A network trained on a large MU-MIMO configuration generalizes to smaller configurations with limited sum-rate loss. The paper also improves fully loaded-system performance through an expanded IAIDNN structure, while noting added complexity and broader future scope.

  • D. Generalization Ability: A network trained with N_t = 256 and K = 80 is applied to scenarios with smaller N_t and K to evaluate generalization.The resulting performance is compared with results from the corresponding smaller scenarios.
  • D. Generalization Ability: Performance loss is around 1% for fewer users with the same transmit-antenna count and around 3% when both users and antennas are fewer.These losses quantify transfer from the large-scale training configuration to smaller scenarios.
  • D. Generalization Ability: The IAIDNN achieves good sum-rate performance when K × N_r < N_t and can outperform classic iterative WMMSE in seven iterations.Its sum-rate performance approaches iterative WMMSE in the reported underloaded setting.
  • E. Improvement of the IAIDNN for Fully Loaded Systems: In a fully loaded system with K × N_r = N_t, the IAIDNN's sum-rate performance degrades.This setting motivates modifying the network structure for improved performance.
  • E. Improvement of the IAIDNN for Fully Loaded Systems: The improved IAIDNN introduces matrix inversion and additional trainable parameters, increasing computational complexity and training requirements slightly.The modified structure replaces A^+X + AY + Z with A^-1X + PAY + Z using trainable X, P, Y, and Z.
  • VII. CONCLUSION: The framework is proposed as a surrogate for iterative optimization algorithms in real-time systems and could be generalized to other communication applications.Future examples include robust precoding with CSI errors, multicell precoding, and problems with discrete variables.

APPENDIX A

The appendix develops the matrix-calculus tools used to derive IAIDNN back-propagation gradients. It establishes gradient formulas and recurrence relations across adjacent network layers.

  • Matrix gradient preliminaries: Theorem 2 gives the partial derivative of a real-valued function with respect to a complex matrix variable from its differential form.The function maps C^(m×n) to R, with X ∈ C^(m×n).
  • Matrix differential preliminaries: Lemma 2 presents matrix-differential properties for constant and variable matrices, supplemented by trace properties.These results support the subsequent matrix-form gradient derivation.
  • Gradient recurrence: The generalized chain rule is obtained from Theorem 2, Lemma 2, and the preceding equation, then applied to objective gradients and adjacent-layer variables.The appendix derives recurrence relations for gradients with respect to W, U, and V across layers, including normalization of V by P^T.
Loading 2006.08099v1…