PaperlayerView exact revision on arXiv

Source-linked AI summary

Performance of Statistical Tests for Single Source Detection using Random Matrix Theory

Pascal Bianchi, Merouane Debbah, Mylène Maïda, Jamal Najim

arXiv:0910.0827v2math.PRcs.ITmath.ST

TL;DR

The paper studies source detection with unknown channel and noise variance, a setting where classical Neyman–Pearson testing cannot directly use known observation densities. It develops a GLRT based on a covariance-eigenvalue statistic, uses random matrix theory for asymptotic threshold and p-value evaluation, and derives error exponents showing asymptotic superiority over a condition-number test.

  • Problem

    Unknown channel and noise variance leave the observation densities under both hypotheses unavailable, requiring analysis of alternative source-detection tests.

  • Method

    The paper analyzes a GLRT based on the ratio of the sampled covariance matrix’s largest eigenvalue to its normalized trace, using random matrix theory and large deviations.

  • Results

    The GLRT admits practical asymptotic threshold and p-value evaluation, closed-form error exponents, and asymptotically outperforms the condition-number test.

  • Takeaways & Limitations

    The framework provides a performance-analysis route for multi-sensor detection when the channel and noise variance are unknown.

Abstract

from arXiv · show

This paper introduces a unified framework for the detection of a source with a sensor array in the context where the noise variance and the channel between the source and the sensors are unknown at the receiver. The Generalized Maximum Likelihood Test is studied and yields the analysis of the ratio between the maximum eigenvalue of the sampled covariance matrix and its normalized trace. Using recent results of random matrix theory, a practical way to evaluate the threshold and the $p$-value of the test is provided in the asymptotic regime where the number $K$ of sensors and the number $N$ of observations per sensor are large but have the same order of magnitude. The theoretical performance of the test is then analyzed in terms of Receiver Operating Characteristic (ROC) curve. It is in particular proved that both Type I and Type II error probabilities converge to zero exponentially as the dimensions increase at the same rate, and closed-form expressions are provided for the error exponents. These theoretical results rely on a precise description of the large deviations of the largest eigenvalue of spiked random matrix models, and establish that the presented test asymptotically outperforms the popular test based on the condition number of the sampled covariance matrix.

I. INTRODUCTION

The paper addresses source detection when the channel and noise variance are unknown, using a GLRT based on the largest covariance eigenvalue relative to normalized trace. In a large-sensor, large-observation regime, it develops practical threshold and p-value approximations and analyzes error exponents, showing asymptotic advantages over a condition-number test.

  • Problem and GLRT: Unknown channel and noise variance make both hypothesis densities unavailable, preventing direct use of the classical Neyman–Pearson test.The paper instead estimates unknown parameters by maximum likelihood, yielding a generalized likelihood ratio test.
  • Problem and GLRT: The GLRT reduces to the ratio of the sampled covariance matrix’s largest eigenvalue to its normalized trace.The model uses K sensors and N received samples, with complex Gaussian noise of unknown variance and an unknown deterministic channel vector.
  • Asymptotic analysis: When K and N are large with the same order of magnitude, random matrix theory provides an asymptotic framework for evaluating the test threshold and p-value.This regime is motivated by applications such as cognitive radio and supports practical threshold evaluation.
  • Performance analysis: Large deviations of the largest eigenvalue in spiked covariance models yield closed-form rate functions and error-exponent expressions for the GLRT.The paper introduces an error-exponent curve as an asymptotic ROC representation in log-log scale.
  • Comparison: The GLRT asymptotically outperforms a competing test based on the sampled covariance matrix’s condition number.The condition-number statistic uses the ratio of the largest to smallest sampled-covariance eigenvalues.

II. GENERALIZED LIKELIHOOD RATIO TEST

This section derives the GLRT for source detection when the channel h and noise variance σ2 are unknown, then reduces it to a test based on the sampled covariance eigenvalues and normalized trace.

  • Observation model: The model stacks N received samples into a K × N matrix Y and forms its sampled covariance matrix ˆR.The noise and signal processes are assumed independent, with noise covariance σ2I_K and unit-variance scalar signal samples.
  • GLRT derivation: The GLRT compares the maximized likelihood under H1, over h and σ2, with the maximized likelihood under H0, over σ2.The likelihoods are defined for the observation matrix Y under the two hypotheses and their unknown parameters.
  • Problem: The GLRT addresses detection when the channel h and noise variance σ2 are unknown, a setting without a simple uniformly most powerful test.The observations comprise K-dimensional Gaussian noise vectors and, under H1, a deterministic channel multiplied by an i.i.d. Gaussian signal.
  • Threshold: The threshold is selected so the false-alarm probability P0(L_N > ξ_N) stays below a prescribed level, then translated into a threshold for T_N.Because the mapping between L_N and T_N is increasing on the relevant interval, the two rejection rules are equivalent.
  • Test statistic: The GLRT is equivalent to rejecting H0 for large values of the statistic T_N, which depends on the largest eigenvalue of ˆR relative to its normalized trace.The ordered eigenvalues λ1 > λ2 > ··· > λK of ˆR enter the closed-form likelihood-ratio expression through T_N.
  • Asymptotic framework: The analysis restricts attention to the traditional GLRT because practical variants replace the normalized trace with more elaborate noise-variance estimates without changing the asymptotic error-exponent analysis.The exact computation raises computational issues, motivating an asymptotic framework in which K and N are large with the same order of magnitude.

B. Exact threshold and p-values

The paper derives exact GLRT threshold and p-value expressions from the sampled covariance eigenvalues, then replaces costly finite-dimensional calculations with asymptotic random-matrix approximations when sensors and snapshots grow together.

  • The exact p-value pN(t) is obtained by integrating the ordered-eigenvalue density over a threshold-dependent domain.
  • The GLRT statistic TN is determined by the largest eigenvalue and normalized trace of the sampled covariance matrix.
  • The exact p-value requires numerical evaluation of a nontrivial integral, which may be impractical for online real-time computation.
  • The proposed asymptotic procedure studies pN as K and N tend to infinity at the same rate, yielding a simpler testing procedure.
  • The asymptotic framework is intended for sensing systems where sensor and sample counts have the same order, including cognitive-radio settings.
  • Under H0, the centered and rescaled largest eigenvalue converges to the Tracy-Widom law, whose tables support threshold and p-value approximations.

B. Behaviour under hypothesis H1

Under H1, the sampled covariance follows a rank-one spiked model. The perturbation leaves the empirical spectral distribution unchanged asymptotically but can move the largest eigenvalue beyond the Marčenko-Pastur support at sufficiently large SNR.

  • Under H1, the covariance has the rank-one spiked form Σ = σ2IK + hh∗, and the sampled covariance follows a spiked random-matrix model.
  • The H1 analysis uses eigenvalue-angle coordinates and spherical integrals to handle the spiked covariance model.
  • The rank-one perturbation does not affect the limiting empirical spectral distribution or its deviations under P1.
  • The largest eigenvalue’s limiting behavior changes when the signal-to-noise ratio ρK is sufficiently large.
  • For a sufficiently large perturbation, the largest eigenvalue converges outside the support of the Marčenko-Pastur distribution.

1) The following convergence holds true:

The paper calibrates the GLRT through Tracy-Widom asymptotics and analyzes its power using large deviations. At fixed level, the resulting error exponent exists and is independent of the chosen level.

  • The p-value associated with the GLRT can be approximated using the Tracy-Widom complementary distribution.
  • The rescaled statistic converges under H0 to a standard Tracy-Widom random variable, enabling asymptotic false-alarm calibration.
  • The threshold γN maximizing power at fixed level α is characterized through Tracy-Widom quantiles.
  • The error exponent is obtained from large deviations of the largest eigenvalue and is defined for fixed testing level.
  • The limit ET exists asymptotically, and ET does not depend on α.
  • The miss probability admits the asymptotic approximation βN,T(α) ≃ e−N ET, with ET depending on c and the limiting signal strength.

B. Large Deviations associated to TN

The performance analysis rests on large-deviation principles for TN and the largest eigenvalue in spiked Wishart models. These rate functions quantify exponentially rare detection errors and require special treatment near discontinuities.

  • Large-deviation analysis formalizes probabilities through rate functions on the scale N, with rare-event probabilities decaying exponentially.
  • The error-exponent results follow from large deviations of TN as both N and K increase.
  • Under H0, TN satisfies an LDP in scale N with a good rate function, and under H1 it has a corresponding rate-function description.
  • The large deviations of TN are driven by the largest eigenvalue because the normalized trace concentrates much faster.
  • Under H1, the eigenvalue joint density includes a spherical-integral term, making the large-deviation analysis harder than under H0.
  • The rate functions require minor modifications for real Gaussian observations because the eigenvalue joint density differs from the complex case.

C. Proof of Theorem 2

The proof characterizes achievable error-exponent pairs for test TN by analyzing threshold sequences and the large deviations of its miss probability. It establishes the pair (I+_0(x), I+_ρ(x)) under suitable thresholds.

  • The proof studies the asymptotic behavior of TN's miss probability to characterize achievable error-exponent pairs.
  • Rejecting H0 when TN > x yields a false-alarm exponent determined by P0(TN > x).
  • The corresponding miss probability is βN,T(αN) = P1(TN < x), whose exponent follows from the large-deviation rate under H1.
  • The resulting achievable pair is (I+_0(x), I+_ρ(x)).
  • The threshold sequence γN is shown to converge to the unique x solving the relevant rate-function condition.
  • The comparison section introduces UN as the condition-number test, whose statistic is λ1/λK and whose alternative limit exceeds λ+/λ− at sufficiently large ρ.

B. A few remarks related to the determination of the threshold for the test UN

This section explains threshold calibration and error-exponent analysis for UN, the condition number of the sampled covariance matrix. It derives rate functions from the joint behavior of λ1 and λK and compares UN with TN.

  • Its threshold can be calibrated using asymptotic independence of the edge fluctuations and quantiles of aX + bY.
  • Under H0, the edge-scaled variables Λ1 and ΛK converge to independent Tracy-Widom random variables.
  • The positive rank-one perturbation leaves λK's rate function unchanged between H0 and H1.
  • UN satisfies a large-deviation principle in scale N with good rate functions under H0 and H1.
  • Although λ1 and λK are not independent, they behave as asymptotically independent variables from a large-deviation perspective.

Appendix B.

Appendix B summarizes the theorem, proof strategy, and numerical evidence for the error-exponent comparison. It reports that TN uniformly dominates UN and is supported by finite-dimensional simulations.

  • Theorem 3: Theorem 3 gives the error-exponent curve SU for UN and states that TN's curve ST uniformly dominates it.
  • Theorem 3: For every achievable pair on SU, there is a pair on ST with the same first exponent and a strictly larger second exponent.
  • Remarks: At fixed level α, both tests' powers converge to one at the same exponential speed, but TN gains when the level decreases exponentially.
  • Numerical results: Figure 2 compares TN's error exponent with the Neyman-Pearson test when all parameters are known, across values of c and ρ.
  • Numerical results: Figure 3 plots the error-exponent curves of TN and UN for c = 1 using the analytic expressions from Theorems 2 and 3.
  • Conclusion: The conclusion emphasizes unknown noise variance and channel, large-deviation analysis, closed-form expressions, and finite-dimension simulation support.
  • Numerical results: Figure 4 reports simulated ROC curves for K = 10, N = 50, and ρ = 10 dB, providing finite-dimensional support for the asymptotic comparison.

ACKNOWLEGMENT

The appendix develops large-deviation arguments for the largest eigenvalue in Wishart and spiked models. It uses spherical-integral asymptotics and leaves some proof details to the reader.

  • Under H0 and H1, the large deviations of λ1 are derived from the eigenvalue densities of unspiked and spiked models.
  • Several technical bounds and parts of the proofs are omitted or left to the reader, including lower-bound arguments and parts of Lemma 1.
  • The spherical integral is asymptotically replaced by an exponential equivalent when analyzing the spiked model.
  • For TN, the trace and empirical eigenvalue measure concentrate faster than λ1, so TN inherits λ1's large-deviation rate function.
  • The resulting rate function for the spiked model is normalized as Gρ − inf_R+ Gρ.

C. Proof of Lemma 1-(3)

The proof analyzes large deviations of T_N through localization and bounds involving g_N(η) and h_N(r). It also uses the N^-2/3 fluctuation scale and Tracy–Widom convergence for the limiting term.

  • The argument requires an additional analysis of the large deviations of T_N.
  • The proof localizes λ_1 relative to g_N(η) and h_N(r), obtaining T_N < g_N(η) under the stated bounds.
  • A lower bound for Q_1(T_N < g_N(η)) combines h_N(r), the increment of g_N, an exponential rate term, and a bulk-eigenvalue event.
  • The factor h_N(r)(g_N(η) − g_N(η − 1)) contributes negligibly at the large-deviation scale.
  • Uniform lower boundedness of the bulk-eigenvalue event makes its normalized logarithm converge to zero, yielding the claimed result.

APPENDIX B

Appendix B develops the joint large-deviation analysis of the largest and smallest eigenvalues, then applies the contraction principle to obtain the large deviations of their ratio. The derivation uses localization, Laplace’s method, and rate-function decompositions under both hypotheses.

  • The appendix studies the joint large deviations of (λ_1, λ_K) before deriving the ratio’s large-deviation principle.
  • The relevant extreme-eigenvalue deviations occur outside the Marčenko–Pastur bulk, with α_1 > λ+ and β_K < λ−, at rate e^-N×const..
  • Bulk eigenvalues deviating from the limiting support are much less likely, occurring at rate e^-N^2×const..
  • The joint probability is approximated by localizing λ_1 and λ_K in intervals A and B and applying Laplace’s method to the remaining integrals.
  • The resulting rate functions account for whether λ− is typical and differ under H0 and H1 through the relevant upper-edge terms.
  • The ratio’s rate function is obtained by minimizing the joint deviation cost over compatible values of λ_1 and λ_K.

APPENDIX C CLOSED-FORM EXPRESSIONS FOR FUNCTIONS f, F+ AND F−

Appendix C records closed-form representations for the Stieltjes transform and the functions F+ and F−. It states the relevant branch conventions, functional identities, and the domain conditions used for these formulas.

  • APPENDIX C CLOSED-FORM EXPRESSIONS FOR FUNCTIONS f, F+ AND F−: The appendix introduces the Stieltjes transform f of the Marčenko–Pastur distribution and collects related standard facts without proofs.
  • APPENDIX C CLOSED-FORM EXPRESSIONS FOR FUNCTIONS f, F+ AND F−: Lemma 4 gives a representation of f together with a system of equations involving f and its companion transform.
  • APPENDIX C CLOSED-FORM EXPRESSIONS FOR FUNCTIONS f, F+ AND F−: The square-root expressions use specified branches, including the principal branch and a branch whose image has nonpositive real part.
  • 4) As a consequence:: Lemma 5 provides closed-form identities for F+ and F−, using the preceding representation and integrating with respect to the Marčenko–Pastur measure.
  • 4) As a consequence:: The formulas apply on domains such as x > λ+, where f and f̃ are holomorphic outside {0} ∪ [λ−, λ+].
  • 4) As a consequence:: The functions f and its companion satisfy system (69), with 1 + cf and 1 + f̃ never vanishing.
  • 4) As a consequence:: Differentiating the system yields logarithmic identities involving c log(1 + cf), log(1 + f̃), and xf f̃.
Loading 0910.0827v2…