PaperlayerView exact revision on arXiv

Source-linked AI summary

AMP-Inspired Deep Networks for Sparse Linear Inverse Problems

Mark Borgerding, Philip Schniter, Sundeep Rangan

arXiv:1612.01183v2cs.IT

TL;DR

The paper targets recovery of sparse signals from a few noisy linear measurements. It unfolds AMP and VAMP into learned networks with Onsager correction, improving accuracy and convergence while making LVAMP robust to ill-conditioned measurement matrices.

  • Problem

    Recovering a sparse signal from a few noisy linear measurements remains the central sparse linear inverse problem addressed by the paper.

  • Method

    The paper unfolds AMP and VAMP into deep networks, jointly learning linear transforms and nonlinear shrinkage functions while incorporating Onsager correction.

  • Results

    After 10 layers, untied LAMP’s NMSE is 0.5 dB from the support-oracle bound and up to 16 dB better than tied LISTA; LVAMP remained robust to ill-conditioned matrices.

  • Takeaways & Limitations

    With i.i.d. signals, LVAMP’s learned parameters nearly match matched-VAMP prescriptions, providing an intuitive MMSE-based interpretation of the network.

  • Takeaways & Limitations

    LAMP performance deteriorated with ill-conditioning in the measurement matrix, whereas LVAMP did not.

Abstract

from arXiv · show

Deep learning has gained great popularity due to its widespread success on many inference problems. We consider the application of deep learning to the sparse linear inverse problem, where one seeks to recover a sparse signal from a few noisy linear measurements. In this paper, we propose two novel neural-network architectures that decouple prediction errors across layers in the same way that the approximate message passing (AMP) algorithms decouple them across iterations: through Onsager correction. First, we propose a "learned AMP" network that significantly improves upon Gregor and LeCun's "learned ISTA." Second, inspired by the recently proposed "vector AMP" (VAMP) algorithm, we propose a "learned VAMP" network that offers increased robustness to deviations in the measurement matrix from i.i.d. Gaussian. In both cases, we jointly learn the linear transforms and scalar nonlinearities of the network. Interestingly, with i.i.d. signals, the linear transforms and scalar nonlinearities prescribed by the VAMP algorithm coincide with the values learned through back-propagation, leading to an intuitive interpretation of learned VAMP. Finally, we apply our methods to two problems from 5G wireless communications: compressive random access and massive-MIMO channel estimation.

I. INTRODUCTION · II. ITERATIVE ALGORITHMS AND DEEP LEARNING · A. Iterative Algorithms

The paper targets sparse linear inverse problems using deep networks informed by AMP and VAMP, with Onsager correction to improve accuracy, complexity, and robustness. It introduces learned AMP/VAMP architectures, jointly learns their transforms and nonlinearities, and applies them to 5G communication problems.

  • I. INTRODUCTION: The problem is to recover a sparse x0 from few noisy linear measurements y, then obtain an estimate of s0 through the known basis Ψ.The formulation uses A ≜ ΦΨ and assumes M ≪ N.
  • I. INTRODUCTION: Existing sparse-recovery methods include sequential and iterative reconstruction algorithms, while deep networks learn to minimize reconstruction MSE from training examples.Once trained, the network predicts a sparse signal for a new measurement y and implicitly depends on the operator and signal/noise statistics.
  • I. INTRODUCTION: Onsager correction from AMP and VAMP is used to modify neural architectures so they require fewer layers for a target accuracy and achieve improved accuracy and complexity.The proposed networks decouple prediction errors across layers in the manner of AMP iterations.
  • I. INTRODUCTION: LAMP unfolds soft-thresholding AMP into a feedforward network with bypass paths, producing outputs more accurate than LISTA at each iteration, in some cases by more than a factor of 10.The bypass paths use a topology and gain-control mechanism distinct from residual and highway networks.
  • I. INTRODUCTION: The work extends LAMP by jointly learning MSE-optimal shrinkage functions and linear transforms, and unfolds VAMP into LVAMP with jointly learned transforms and shrinkage functions.For i.i.d. signals, learned LVAMP parameters are nearly identical to matched-VAMP parameters.
  • I. INTRODUCTION: The proposed networks are applied to compressive random access and massive-MIMO channel estimation in 5G wireless communications.These applications are presented as two problems arising in fifth-generation wireless systems.
  • Notation:: The notation defines norms, diagonal matrices, identity and zero vectors, probability distributions, expectations, variances, Gaussian densities, and the signum function.These conventions support the mathematical development of the algorithms.
  • A. Iterative Algorithms: A standard algorithmic route solves the convex ℓ1 problem, whose tunable λ controls the tradeoff between sparsity and measurement fidelity and whose convexity yields convergence guarantees and performance bounds.ISTA is introduced as one of the simplest approaches for solving this problem and iterates from bx0 = 0.

1) ISTA: · 2) FISTA: · 3) AMP:

ISTA uses a stepsize and componentwise soft-thresholding, while FISTA accelerates convergence by requiring roughly an order of magnitude fewer iterations. AMP adds Onsager correction and prescribed iteration-dependent thresholds, yielding decoupled denoising under suitable random matrices but instability under deviations from that model.

  • 1) ISTA:: ISTA uses a stepsize β and componentwise soft-thresholding shrinkage ηst(·; λ) to process the residual measurement error.The iteration-t residual measurement error is denoted vt.
  • 2) FISTA:: FISTA was introduced as a modification of ISTA because ISTA converges somewhat slowly.The passage identifies FISTA as one of the most famous speed-up modifications.
  • 2) FISTA:: Roughly an order of magnitude fewer iterations are required by FISTA than by ISTA.This comparison is reported with reference to Fig. 1.
  • 3) AMP:: AMP-ℓ1 differs from ISTA by including an Onsager correction term btvt−1 in its residual vt.The correction is identified as the first major difference between AMP-ℓ1 and ISTA.
  • 3) AMP:: AMP-ℓ1 uses a prescribed, iteration-dependent shrinkage threshold λt, with tuning parameter α corresponding one-to-one with λ.The threshold specification is identified as AMP’s second major difference from ISTA.
  • 3) AMP:: AMP can use any Lipschitz-continuous shrinkage function, including noise-standard-deviation estimates σt as an argument.With η(rt; σt, α) = ηst(rt; ασt) and θt = α, the generalized form reduces to AMP-ℓ1.
  • 3) AMP:: For typical large i.i.d. sub-Gaussian matrices, the Onsager correction decouples AMP iterations so the shrinkage input behaves as AWGN-corrupted x0 with known variance σ2t.This behavior is stated for η with identical scalar components and is supported by numerical evidence in Fig. 5(b).
  • 3) AMP:: Small deviations from the i.i.d. sub-Gaussian matrix model can make AMP diverge or behave in poorly understood ways.The paper contrasts this limitation with AMP’s well-understood behavior under the i.i.d. model.

4) Vector AMP:

VAMP preserves AMP’s low complexity, rapid convergence, and AWGN-like shrinkage inputs while extending robustness to large right-rotationally invariant measurement matrices. Its two alternating estimator stages use Onsager corrections to decouple iterations, and VAMP-ℓ1 empirically converges faster than the compared methods, including when AMP-ℓ1 diverges.

  • Vector AMP: VAMP extends AMP’s desirable properties to large right-rotationally invariant measurement matrices, including low per-iteration complexity, few iterations to convergence, and AWGN-modeled shrinkage inputs.I.i.d. Gaussian matrices are a special case, and VAMP remains well behaved for arbitrary orthogonal factors and singular values when dimensions are sufficiently large.
  • Vector AMP: VAMP alternates an LMMSE stage with componentwise nonlinear shrinkage, applying divergence computation, Onsager correction, and variance computation in each stage.The Onsager terms decouple the stages and iterations, enabling local MSE optimization to support global MSE optimization.
  • Vector AMP: With an i.i.d. signal and MMSE denoiser, VAMP’s fixed-point MSE agrees with the replica prediction of MMSE.The paper calls this configuration matched VAMP when the noise variance is known.
  • Comparison of ISTA, FISTA, AMP-ℓ1, and VAMP-ℓ1: VAMP-ℓ1 required about half the iterations of AMP-ℓ1 for i.i.d. Gaussian measurements.In the ill-conditioned scenario, AMP-ℓ1 diverged while VAMP-ℓ1 required an order-of-magnitude fewer iterations than FISTA, which required an order-of-magnitude fewer than ISTA.

B. Deep Learning

Deep networks process observations through repeated linear transformations and componentwise nonlinearities, and can accurately predict continuous, high-dimensional sparse signals. Unfolding ISTA produced LISTA, whose learned layer parameters achieved comparable MSE with far fewer matrix-vector multiplications than classical methods.

  • Deep Learning: Deep networks apply repeated linear transformations followed by simple componentwise nonlinearities to predict unknown labels from observed features.In sparse linear inverse problems, the labels are continuous and high-dimensional, yet deep networks can predict them accurately.
  • Learned ISTA: Unfolding T iterations of ISTA yields a T-layer feed-forward network, establishing the architecture used by learned ISTA.The unfolded network is shown as a feed-forward neural network with one layer corresponding to each ISTA iteration.
  • Learned ISTA: LISTA learns layer-dependent thresholds and matrices from training data instead of using ISTA’s fixed transforms and common threshold.Its learnable parameters are Θ = [B, S, λ], and the network output depends on these parameters.
  • Learned ISTA: 16 layers reached NMSE −35 dB, compared with 25 AMP-ℓ1 iterations, 216 FISTA iterations, and 4402 ISTA iterations.For the i.i.d. Gaussian problem, LISTA achieved comparable MSE with significantly fewer matrix-vector multiplications than existing algorithms with optimally tuned regularization parameters.

III. LEARNED AMP-ℓ1 · A. The LAMP-ℓ1 Network

The paper unfolds AMP-ℓ1 into the learned AMP-ℓ1 (LAMP-ℓ1) network and learns MSE-optimal parameters. LAMP-ℓ1 uses structured, potentially layer-dependent transforms, an Onsager bypass, and realization-dependent shrinkage thresholds to improve on LISTA.

  • III. LEARNED AMP-ℓ1: The structured S has 2MN free parameters and is advantageous in memory and training over unstructured S when M < N/2.The condition is noted as often holding in compressive sensing.
  • III. LEARNED AMP-ℓ1: The structured transform uses B ∈ R^N×M and A ∈ R^M×N, yielding layers initialized with b x_0 = 0 and v_0 = y.This structure is the form suggested by equation (26).
  • III. LEARNED AMP-ℓ1: Allowing A_t and B_t to vary across layers provides a modest performance improvement over fixed transforms.This improvement costs a T-fold increase in memory and training complexity.
  • III. LEARNED AMP-ℓ1: Networks with fixed A and B across layers are called tied, while networks with layer-dependent A_t and B_t are called untied.The tied and untied definitions distinguish transform-sharing choices across the T layers.
  • A. The LAMP-ℓ1 Network: LAMP-ℓ1 is constructed by unfolding AMP-ℓ1 iterations and learning the network parameters to minimize mean squared error.The authors hope this requires fewer layers than LISTA, paralleling AMP-ℓ1’s iteration advantage over ISTA when A is i.i.d. Gaussian.
  • A. The LAMP-ℓ1 Network: Each LAMP-ℓ1 layer has learnable parameters A_t, B_t, and α_t.Figure 4 depicts the tth layer with these three learnable quantities.
  • A. The LAMP-ℓ1 Network: LAMP-ℓ1 includes a bypass path from v_t to v_t+1 that implements an Onsager correction absent from LISTA.Its goal is to decouple network layers in the same way Onsager correction decouples AMP iterations.
  • A. The LAMP-ℓ1 Network: LAMP-ℓ1’s shrinkage threshold λ_t = α_t∥v_t∥_2/√M varies with v_t, whereas LISTA’s threshold does not.The threshold therefore depends on the current realization of the layer state.

B. Parameterizing LAMP-ℓ1 · C. Learning the LAMP-ℓ1 Parameters

LAMP-ℓ1 generalizes AMP-ℓ1 by learning layer-dependent linear transforms while preserving Gaussian shrinkage-input errors under i.i.d. Gaussian measurements. Its parameters are learned with hybrid optimization, bootstrapping, and optional measurement-matrix estimation.

  • B. Parameterizing LAMP-ℓ1: LAMP-ℓ1 generalizes AMP-ℓ1 by replacing fixed (A, A^T) with layer-dependent transforms (A_t, B_t).AMP-ℓ1 enforces B_t = A^T and A_t = A_0 for every layer, whereas LAMP-ℓ1 does not.
  • B. Parameterizing LAMP-ℓ1: Under β_t > 0 and i.i.d. Gaussian A, numerical results indicate that LAMP-ℓ1 preserves AMP’s independent-Gaussian shrinkage-input error behavior.This behavior is identified as AMP’s most important feature.
  • B. Parameterizing LAMP-ℓ1: The parameterization uses the true measurement matrix A when known and can exploit fast matrix-vector multiplication, such as an FFT implementation.When A is unknown, an estimate computed from training data can be substituted.
  • B. Parameterizing LAMP-ℓ1: AMP-ℓ1 and LAMP-ℓ1 shrinkage-input errors are Gaussian, whereas ISTA’s errors are heavy-tailed, as shown by QQ plots at matched recovery stages.The comparison uses the first layer reaching NMSE(bx_t) < −15 dB: t = 1478 for ISTA, t = 6 for AMP-ℓ1, and t = 3 for untied LAMP-ℓ1.
  • C. Learning the LAMP-ℓ1 Parameters: Joint back-propagation from B = A^T, α_0 = 1, and β_0 = 1 converged to a bad local minimum, plausibly because B had 125 000 free parameters.The authors therefore propose a hybrid layer-wise and global optimization strategy.
  • C. Learning the LAMP-ℓ1 Parameters: For tied LAMP-ℓ1, Algorithm 2 alternates layer-wise learning of the current α_t and β_t with global learning of all parameters through layer t.β_0 remains fixed because {B, α_0, β_0} is over-parameterized.
  • C. Learning the LAMP-ℓ1 Parameters: Untied LAMP-ℓ1 uses bootstrapping to avoid bad local minima, reinitializing from tied parameters when its performance is inappropriate at a layer.The method is motivated by the expectation that choosing a different B_t per layer should perform at least as well as using one shared B.
  • C. Learning the LAMP-ℓ1 Parameters: Least-squares estimation of unknown A followed by joint optimization produces essentially no difference in final test MSE compared with training using known A.This empirical result comes from experiments not detailed in the paper.

D. Discussion … 1) Scaled Soft-Threshold:

The paper develops LAMP by adding Onsager correction to learned iterative networks, learning layer-specific linear transforms and shrinkage parameters. It also constrains and generalizes the shrinkage functions, including scaled soft-thresholding as a special case of the general LAMP network.

  • D. Discussion: LAMP-ℓ1 differs from LISTA by Onsager correction and outperforms LISTA at every layer in the reported numerical experiments.The comparison uses the numerical data that produced Fig. 1.
  • IV. LEARNED AMP: Learned AMP seeks jointly MSE-optimal shrinkage functions and linear transforms across all network layers.The optimization uses finite-dimensional families of shrinkage functions and back-propagation to learn their parameters and the linear-transform parameters.
  • IV. LEARNED AMP: The general LAMP layer has learnable parameters B_t and θ_t.Figure 6 illustrates the tth layer of the general LAMP network.
  • A. The LAMP Network: LAMP unfolds generic AMP into a network with layer-dependent learnable transform pairs (A_t, B_t) and restrictions on A_t that facilitate Onsager correction.This generalizes the fixed transform pair (A, A^T) used in AMP.
  • 1) Scaled Soft-Threshold:: In AMP-ℓ1, β_t acts to scale the soft-thresholding output, motivating its role among the tunable parameters in LAMP.The LAMP construction relaxes the transform pair while retaining restrictions needed for Onsager correction.
  • B. Parameterizing the Shrinkage Functions: The shrinkage families considered for LAMP are separable and odd symmetric.Separable shrinkage acts componentwise, while odd symmetry requires η(r; σ, θ) = −η(−r; σ, θ).
  • 1) Scaled Soft-Threshold:: The scaled soft-threshold family uses θ ∈ R^2, and LAMP-ℓ1 is a special case with η = η_sst and θ_t = [β_t, α_t].The scaled soft-threshold operator is recognized as a scaled version of the soft-threshold operator.

2) Piecewise Linear:  … 5) Bernoulli-Gaussian:

The paper develops several learnable shrinkage families, progressing from five-segment piecewise linear functions through exponential and spline forms to MSE-optimal Bernoulli-Gaussian denoisers. Their parameters control breakpoints, slopes, transition rates, noise dependence, or prior parameters.

  • 2) Piecewise Linear: : The piecewise linear family uses five odd-symmetric segments, generalizing the three-segment form while allowing a potentially nonzero middle slope.Its five parameters determine four vertices at [−θ2σ, −θ1σ, θ1σ, θ2σ] and segment slopes [θ5, θ4, θ3, θ4, θ5].
  • 3) Exponential:: The exponential shrinkage family is introduced as another parameterized shrinkage design.Its three parameters control asymptotic and origin slopes plus transition rate, with dependence on estimated noise level σ.
  • 3) Exponential:: In the exponential family, θ2 sets the asymptotic slope, θ2 + θ3 sets the origin slope, and larger θ1 produces a slower transition.The design is inspired by [37, Eq. (19)-(20)] and depends on the estimated noise level σ.
  • 4) Spline: : The spline shrinkage family is introduced as a separate compact parameterization.Its definition includes a cubic segment proportional to 6(2 −|z|)3 for 1 ≤|z| ≤2 and zero for 2 ≤|z|.
  • 4) Spline: : Spline parameters control asymptotic slope θ2, origin slope θ2 + 2θ3, and transition rate, with larger θ1 yielding a slower transition.The three-parameter form depends on σ and contrasts with a prior construction using 8000 shifted basis functions and adjustable weights.
  • 5) Bernoulli-Gaussian:: The Bernoulli-Gaussian family uses MSE-optimal denoisers, defining bx = E{x|r} for a zero-mean BG prior and an AWGN-corrupted measurement r.The prior has γ ∈(0, 1) and φ > 0.
  • 5) Bernoulli-Gaussian:: The BG denoiser is converted into a learnable shrinkage function by setting θ1 = φ and θ2 = log 1−γ.These parameter assignments tie the learnable form to the BG prior parameters.

C. Learning the LAMP Parameters … C. Discussion

The paper unfolds AMP and VAMP into learnable networks, comparing tied and untied parameterizations and showing that LVAMP improves robustness and interpretability. LAMP performs strongly for i.i.d. Gaussian matrices but degrades for non-i.i.d. and ill-conditioned matrices.

  • C. Learning the LAMP Parameters: LAMP uses tied or untied linear transforms across layers, with parameters learned by replacing the standard iteration parameters with θ_t.The tied case shares one transform across layers; the untied case permits a different transform in each layer.
  • D. Discussion: After 10 layers, untied LAMP reaches an NMSE 0.5 dB from the support-oracle bound and up to 16 dB beyond tied LISTA.LAMP is constructed by unfolding AMP, parameterizing its linear and nonlinear stages, and learning them through hybrid layer-wise and global optimization.
  • D. Discussion: LAMP performance suffers for non-i.i.d.-Gaussian and especially ill-conditioned A, while its learned parameters remain difficult to interpret.These issues arise because LAMP replaces AMP’s (A, A^T) with layer-dependent (β_tA, B_t).
  • V. LEARNED VECTOR-AMP: VAMP motivates a learned network because it preserves AMP’s benefits for the broader class of right-rotationally invariant measurement matrices.AMP can diverge or become poorly understood under even small deviations from i.i.d. sub-Gaussian matrices.
  • A. The LVAMP Network: Each LVAMP layer performs LMMSE estimation, decoupling, shrinkage, and a second identical decoupling stage, with learnable LMMSE and shrinkage parameters.The LMMSE stage can use SVD-based parameters for i.i.d. signals or unconstrained matrices for general signals.
  • B. Learning the LVAMP Parameters: LVAMP parameters are learned with separate tied and untied procedures, using SVD-based initialization when A is known and an estimated-A SVD otherwise.The suggested initialization also sets σ^2 across the training data.
  • C. Discussion: With i.i.d. signals, tied and untied LVAMP perform near-identically, LVAMP is more robust than LAMP to A, and it converges slightly faster for i.i.d. Gaussian A.Learned LVAMP parameters are essentially identical to matched VAMP values, preserving its interpretable alternation of linear vector and nonlinear scalar MMSE estimation.

D. Summary of Computational and Memory Complexity · VI. NUMERICAL INVESTIGATION · A. Effect of Onsager Correction and Parameter Learning

The paper compares the computational and memory costs of LISTA, LAMP, and LVAMP, then evaluates how Onsager correction, parameter learning, shrinkage choice, and network untying affect recovery across measurement matrices. LAMP-ℓ1 accelerates convergence over LISTA and AMP-ℓ1, while LVAMP and learned shrinkage improve robustness or accuracy under non-i.i.d. matrices.

  • D. Summary of Computational and Memory Complexity: Untied LISTA has computational and memory complexities ≈T N^2, while tied LISTA reduces memory complexity to N^2.Both versions learn B, shrinkage parameters, and layer transforms, but tied LISTA stores only one S.
  • D. Summary of Computational and Memory Complexity: Untied LAMP requires computational complexity ≈2T NM and memory complexity ≈T MN, while tied LAMP reduces memory complexity to MN.The tied version learns only one B instead of one B_t per layer.
  • D. Summary of Computational and Memory Complexity: For i.i.d. signals with SVD-parameterized parameters, LVAMP needs memory complexity ≈T |θ| and computational complexity ≈2T NM; alternative (G_t, H_t) parameters cost ≈T N^2 for both.The SVD quantities are known and need not be learned or stored; fast implementations of A and V can further reduce LAMP and LVAMP computation.
  • VI. NUMERICAL INVESTIGATION: The numerical investigation trains and tests sparse Bernoulli-Gaussian signals at 40 dB SNR using 250×500 measurement matrices that are either i.i.d. Gaussian or have condition number 15.Training used mini-batches of size D = 1000 with Adam and TensorFlow.
  • A. Effect of Onsager Correction and Parameter Learning: At NMSE = −34 dB, tied LAMP-ℓ1 required 7 layers, compared with 15 for tied LISTA and 25 iterations for AMP-ℓ1 under i.i.d. Gaussian A.Tied LAMP-ℓ1 significantly outperformed both alternatives at each layer, isolating the effects of Onsager correction and parameter learning.
  • A. Effect of Onsager Correction and Parameter Learning: For κ = 15, AMP-ℓ1 diverged while LAMP-ℓ1 remained stable and retained roughly the same advantage over tied LISTA as with i.i.d. Gaussian A.Untying yielded improvements of ≤2 dB for i.i.d. Gaussian A and ≤3 dB for κ = 15, at the cost of T-fold parameter storage and longer training.
  • A. Effect of Onsager Correction and Parameter Learning: For κ = 15, BG shrinkage dominated untied LAMP at all layers and came within 1 dB of the support-oracle bound for t ≥13, while tied piecewise-linear shrinkage performed significantly better after ≥10 layers.These experiments indicate predictable behavior for i.i.d. Gaussian matrices but less predictable behavior for non-i.i.d.-Gaussian matrices; with κ = 100, tied LAMP converged much more slowly and BG and piecewise-linear versions separated substantially.

C. LVAMP’s Robustness to the Matrix A · D. Equivalence of LVAMP and Matched VAMP

LVAMP remains robust as the measurement matrix becomes ill-conditioned, outperforming LAMP in shallow networks and matching its deeper performance. Across tested matrices, LVAMP achieves NMSE indistinguishable from matched VAMP, while learned parameters closely follow matched-VAMP prescriptions even with flexible piecewise-linear shrinkage.

  • C. LVAMP’s Robustness to the Matrix A: LVAMP is robust to ill-conditioned measurement matrices, motivating its evaluation under both BG and piecewise-linear shrinkage.The experiments focus on these families because BG matches the signal model, while piecewise-linear shrinkage provides flexibility without prior-distribution knowledge.
  • C. LVAMP’s Robustness to the Matrix A: LVAMP’s tied and untied versions produce essentially identical NMSE across i.i.d.-Gaussian A and A with condition number κ = 15.The corresponding learned parameters are also nearly identical at every layer.
  • C. LVAMP’s Robustness to the Matrix A: 2 dB better NMSE than tied LAMP is achieved by LVAMP for networks with > 4 layers under i.i.d.-Gaussian A, for both shrinkage families.LVAMP is also noticeably better than untied LAMP at 4-8 layers.
  • C. LVAMP’s Robustness to the Matrix A: With > 10 layers, LVAMP and untied LAMP perform equally well and remain within 0.5 dB of the support-oracle bound.This comparison applies to the i.i.d.-Gaussian measurement-matrix experiment.
  • D. Equivalence of LVAMP and Matched VAMP: LVAMP networks achieve NMSE indistinguishable from matched VAMP for all tested measurement matrices.Matched VAMP is evaluated under statistically matched i.i.d. signal and noise models.
  • D. Equivalence of LVAMP and Matched VAMP: LVAMP-BG13 learns parameters that coincide almost perfectly with those prescribed by matched VAMP.The paper describes matched VAMP as predicting the parameters learned by back-propagation.
  • D. Equivalence of LVAMP and Matched VAMP: Matched VAMP explains LVAMP parameters through alternating MSE-optimal linear and nonlinear estimation under Onsager-generated pseudo-priors and pseudo-measurements.Its iteration includes vector estimation, Onsager decoupling, scalar estimation, and a second Onsager decoupling stage.
  • D. Equivalence of LVAMP and Matched VAMP: Piecewise-linear LVAMP performs just as well as matched VAMP without using knowledge of the prior on x0 beyond i.i.d. structure.The prior-family knowledge used by both LVAMP-BG and matched VAMP makes their agreement less practically significant.

VII. APPLICATION TO 5G COMMUNICATIONS … VIII. CONCLUSION

LAMP and LVAMP were applied to compressive random access and massive-MIMO channel estimation, where they outperformed LISTA and CNNs in the reported experiments. The conclusion attributes LAMP’s design to unfolded AMP with Onsager correction and presents LVAMP as more robust to ill-conditioned matrices, while proposing generalized-model extensions.

  • VII. APPLICATION TO 5G COMMUNICATIONS: The proposed LAMP and LVAMP networks were evaluated on compressive random access and massive-MIMO channel estimation, both formulated as sparse linear inverse problems.LVAMP used the LMMSE parameterization (41).
  • A. Application to Compressive Random Access: Sporadic user activity makes the activity/channel vector sparse, enabling sparse recovery for joint user-activity detection and channel estimation.For N = 512 users and activity rate γ = 0.01, the analysis suggests pilots of length M ≥64.
  • A. Application to Compressive Random Access: LAMP and LVAMP significantly outperformed tied and untied LISTA in compressive random access; piecewise linear shrinkage performed about 0.5 dB better than BG shrinkage.Untied LVAMP performed about 0.5 dB better than untied LAMP, while untied LAMP performed about 0.5 dB better than tied LAMP.
  • B. Application to Massive-MIMO Channel Estimation: For massive-MIMO channel estimation, piecewise-linear shrinkage ranked LVAMP, untied LAMP, tied LAMP, untied LISTA, and tied LISTA from best to worst at 6 layers.Piecewise-linear shrinkage outperformed BG shrinkage by roughly 0.5 dB.
  • C. Discussion: CNNs did not obtain good results on the two 5G problems, which the authors relate to their limited structure beyond sparsity.The authors contrast this with CNN effectiveness for richly structured signals such as images.
  • VIII. CONCLUSION: LAMP unfolds AMP into a deep network, learning parameters from training data while adding Onsager correction paths and jointly learning linear transforms and nonlinearities.These mechanisms distinguish LAMP from LISTA.
  • VIII. CONCLUSION: LAMP and LVAMP significantly outperformed LISTA in convergence rate and final MSE; LAMP deteriorated under ill-conditioning in A, whereas LVAMP did not.With i.i.d. signals, learned LVAMP parameters were nearly identical to those prescribed by matched VAMP.
  • VIII. CONCLUSION: The authors report gains over LISTA and conventional deep CNNs in both 5G applications and propose unfolding generalized AMP or VAMP for generalized linear models.Potential applications include phase retrieval and quantized compressive sensing.

APPENDIX A DERIVATION OF LAMP-ℓ1 EQUATIONS (30) · APPENDIX B 5G CHANNEL MODELING DETAILS

Appendix A derives the LAMP-ℓ1 layer equations by rescaling the soft-thresholding parameters and redefining variables. Appendix B specifies a multipath 5G channel model, transforms it into a DFT-domain sparse linear model, and identifies the sparse inverse-problem special case.

  • APPENDIX A DERIVATION OF LAMP-ℓ1 EQUATIONS (30): The soft thresholder’s scaling identity permits rewriting the LAMP-ℓ1 layer equations with rescaled threshold parameters.The identity is ηst(r; λ) = ηst(βr; βλ)/β for any β > 0.
  • APPENDIX A DERIVATION OF LAMP-ℓ1 EQUATIONS (30): Defining λt ≜βtλt and αt ≜βtαt yields the parameterization used for the tth LAMP layer.The appendix states that these definitions, together with the earlier definitions of λt and bt, imply the layer implementation.
  • APPENDIX A DERIVATION OF LAMP-ℓ1 EQUATIONS (30): The αt parameters are freely adjustable, and equation (47) is rewritten as equation (30) by redefining bxt and dropping bars from the remaining variables.This redefinition avoids excess notation in the main paper.
  • APPENDIX B 5G CHANNEL MODELING DETAILS: The 5G system model covers compressive random access and massive-MIMO channel estimation as special cases.It considers Nc nearby cells, up to Nu single-antenna users per cell, and a base station with Nr antennas using a half-wavelength-spaced uniform linear array.
  • APPENDIX B 5G CHANNEL MODELING DETAILS: Active users simultaneously transmit pilot waveforms, while each base station measures their multipath propagation across discrete times and antennas.The model includes activity indicators, pilot waveforms, path gains, delays, arrival angles, sampling interval T, noise, and residual far-cell interference.
  • APPENDIX B 5G CHANNEL MODELING DETAILS: The narrowband approximation replaces an(mT−τnp) with an(mT) when τnp ≪T, enabling elementwise definitions of A and Z and a reduced matrix model.The waveforms are assumed approximately bandlimited to T−1 Hz.
  • APPENDIX B 5G CHANNEL MODELING DETAILS: Representing user channels in critically sampled receive directions gives Z = XF using a DFT matrix F, after which transformed measurements produce a linear model.The coefficient xnl represents user n’s contribution to receive direction l; columns of X become more sparse as Nr grows when users occupy at most D directions.
  • APPENDIX B 5G CHANNEL MODELING DETAILS: Restricting to one receive angle, or using a single-antenna base station, yields y = Ax + w, the sparse linear inverse problem used in the paper.The experiments use Pn = 5 paths, angle spread 10◦, Rician fading with k-factor 10, and path-loss exponent ρ = 4.
Loading 1612.01183v2…