PaperlayerView exact revision on arXiv

Source-linked AI summary

Parameter Estimation For Multivariate Generalized Gaussian Distributions

F. Pascal, L. Bombrun, J. Y. Tourneret, Y. Berthoumieu

arXiv:1302.6498v2stat.AP

TL;DR

Estimating MGGD parameters is important for modeling image data, and the paper addresses whether the scatter-matrix maximum-likelihood estimator is well-defined. It profiles out the scale parameter and analyzes a fixed-point approach; the scatter-matrix MLE exists and is unique up to scale for β ∈ (0, 1), with the fixed-point sequence converging to a maximum.

  • Problem

    The paper addresses estimation of MGGD parameters, which are used to model image statistics such as wavelet and curvelet coefficients.

    Source: §I · p. 1

  • Method

    The paper profiles the scale parameter from the likelihood and analyzes a fixed-point equation for estimating the scatter matrix.

    Source: B. Contributions · p. 4, §VI · p. 8

  • Results

    For β ∈ (0, 1), the scatter-matrix MLE exists and is unique up to a scalar factor, and the fixed-point sequence converges to a point where the likelihood reaches its maximum.

    Source: B. Contributions · p. 4, §VI · p. 8

  • Takeaways & Limitations

    The results support maximum-likelihood estimation of the MGGD scatter matrix within the stated β range.

    Source: B. Contributions · p. 4, §VI · p. 8

Abstract

from arXiv · show

Due to its heavy-tailed and fully parametric form, the multivariate generalized Gaussian distribution (MGGD) has been receiving much attention for modeling extreme events in signal and image processing applications. Considering the estimation issue of the MGGD parameters, the main contribution of this paper is to prove that the maximum likelihood estimator (MLE) of the scatter matrix exists and is unique up to a scalar factor, for a given shape parameter \beta\in(0,1). Moreover, an estimation algorithm based on a Newton-Raphson recursion is proposed for computing the MLE of MGGD parameters. Various experiments conducted on synthetic and real data are presented to illustrate the theoretical derivations in terms of number of iterations and number of samples for different values of the shape parameter. The main conclusion of this work is that the parameters of MGGDs can be estimated using the maximum likelihood principle with good performance.

I. INTRODUCTION

MGGDs are used to model image features and other signal data, but estimating their parameters requires methods suited to their distributional structure. The paper establishes MGGD-specific properties of the scatter-matrix estimator and introduces the distribution and stochastic representation used in the analysis.

I. INTRODUCTION

  • MGGDs model image statistics, including wavelet and curvelet coefficients, and have been used in denoising, retrieval, thresholding, and texture classification.The cited applications also include radar, video processing, and biomedical signal processing.Source: §I · p. 1
  • For β ∈ (0, 1), the scatter-matrix MLE exists and is unique up to a scalar factor, and a Newton–Raphson procedure is proposed to compute it.The paper distinguishes its analysis from prior SIRV fixed-point results.Source: §I · p. 1

A. Definitions

  • The paper defines an MGGD using a scatter matrix and scale and shape parameters; β = 1 gives the multivariate Gaussian, while β tending to infinity yields a multivariate uniform distribution.The scatter matrix is normalized to have trace p.Source: A. Definitions · p. 2

B. Stochastic representation

  • The stochastic representation expresses an MGGD vector using a uniformly distributed direction and a positive scalar random variable.The scalar variable’s distribution is specified in the passage, but its formula is not needed here.Source: B. Stochastic representation · p. 2

C. MGGD parameter estimation for known β

For known β ∈ (0, 1), the paper derives likelihood-based estimators of the scatter matrix and scale. It characterizes their fixed-point equation and notes where existing M-estimator results do not apply directly.

C. MGGD parameter estimation for known β

  • With β known in (0, 1), the scatter-matrix estimator satisfies a fixed-point equation, and the scale parameter can be profiled out of the likelihood.The scatter matrix can be estimated independently of the scale parameter.Source: C. MGGD parameter estimation for known β · p. 2
  • The fixed-point equation is close to the sample covariance estimator when β = 1 and reduces to the fixed-point covariance estimator when β = 0.The paper adopts a trace constraint because solutions are defined up to a scale factor.Source: C. MGGD parameter estimation for known β · p. 2
  • Maronna’s M-estimator properties cannot be applied directly because the relevant function is discontinuous around zero.The paper says the function otherwise roughly satisfies Maronna’s conditions for β ∈ (0, 1).Source: C. MGGD parameter estimation for known β · p. 3

D. MGGD parameter estimation for unknown β

For unknown β, the likelihood relations estimate β and the scatter matrix independently of the scale parameter. The surrounding analysis characterizes the fixed-point formulation and states the assumption used for its subsequent results.

D. MGGD parameter estimation for unknown β

  • When β is unknown, the likelihood equations jointly estimate the scatter matrix, scale, and shape, while the scatter matrix and β can be estimated independently of scale.The β relation uses the digamma function.Source: D. MGGD parameter estimation for… · p. 3

III. STATEMENTS OF THE MAIN RESULTS

  • The fixed-point results are stated for known β ∈ (0, 1), while the unknown-β case is left for discussion in the simulation section.The analysis describes the expected existence, uniqueness up to scale, and convergence properties for the known-β estimator.Source: §III · p. 3
  • Maronna’s function depends on an individual sample, whereas the function in this analysis depends on all samples.This distinction is noted alongside the discussion of Maronna’s conditions.Source: §III · p. 3

A. Notations

  • The profile likelihood Fχ is homogeneous of degree zero, whereas the fixed-point function fχ is homogeneous of degree one.Fχ is the likelihood after replacing scale with its estimator.Source: A. Notations · p. 3
  • The results rely on hypothesis (H): every selection of p sample vectors is linearly independent.The passage identifies this as a key assumption for the subsequent results.Source: A. Notations · p. 4

B. Contributions

For β∈(0,1), the paper establishes existence and scale-wise uniqueness of fixed points, and convergence of the iteration to a likelihood maximum. It also describes a normalized iteration for practical computation.

Theorem III.1

  • For β∈(0,1), each positive norm α has a unique fixed point αc_MFP, and Fχ reaches its maximum along c_MFP’s positive half-line.Theorem III.1 identifies the unique norm-one representative and the maximum’s half-line.Source: B. Contributions · p. 4
  • The proof establishes a maximum by continuously extending Fχ as zero on the boundary of D, except at the zero matrix, where the function is positive and bounded.The paper concludes that Fχ reaches its maximum in D.Source: B. Contributions · p. 4

Theorem III.2

  • From every initial M0∈D, the recursion converges to a fixed point where Fχ reaches its maximum.Theorem III.2 describes the iteration as a discrete dynamical system on D.Source: B. Contributions · p. 4

Corollary III.1

  • Because the recursion does not control the fixed point’s norm, the paper proposes applying M-normalization at each iteration for practical convenience.The paper also states that the resulting sequence converges to c_MFP up to a scaling factor.Source: B. Contributions · p. 4

IV. PROOF OUTLINE

The section outlines proofs of Theorems III.1 and III.2 through lemmas and propositions, with detailed arguments deferred to appendices.

IV. PROOF OUTLINE

  • The section’s purpose is to provide proofs of Theorems III.1 and III.2.Each proof is decomposed into lemmas and propositions whose arguments are postponed to appendices.Source: §IV · p. 4
  • Proofs directly obtainable from are referred to that work.The passage says the differences require only slight modifications.Source: §IV · p. 4
  • The slight modifications reflect the paper’s fχ definition and its MGGD model for observed vectors x_i.The passage specifies vectors x_i for i = 1, ..., N.Source: §IV · p. 4

A. Proof of Theorem III.1

The proof outline establishes existence and uniqueness for Theorem III.1, then addresses convergence for Theorem III.2; the simulation setup uses repeated Monte Carlo runs.

A. Proof of Theorem III.1

  • Proposition IV.1 establishes a fixed point, while Proposition IV.2 and results from yield a unique norm-one matrix satisfying (9).The existence argument identifies a fixed point at which Fχ reaches its supremum; the proof also seeks to show all fixed points are collinear.Source: A. Proof of Theorem III.1 · p. 5

B. Proof of Theorem III.2

  • To prove convergence, the argument uses compact invariant limit sets and Proposition IV.3’s eventual strict increase property.A bounded orbit’s cluster-point set is compact and invariant; convergence follows when that set reduces to one point.Source: B. Proof of Theorem III.2 · p. 5

V. SIMULATIONS

  • The simulation experiments use 1000 Monte Carlo runs to evaluate the proposed estimators on generated MGGD data.The first scenario uses i.i.d. p-dimensional vectors generated by the stochastic representation (3).Source: §V · p. 5

A. Influence of the normalization

Scatter-matrix normalization leaves the MLE’s statistical performance unchanged while significantly speeding convergence. The estimator’s bias and consistency remain small across tested shape parameters, and the unknown-shape algorithm combines scatter-matrix estimation with Newton–Raphson updates.

A. Influence of the normalization

  • Normalization preserves estimation performance while significantly accelerating convergence; the variance of β̂ remains close to the CRLB with or without normalization.The comparisons cover bias and consistency versus sample size and β̂ variance versus sample size.Source: A. Influence of the normalization · pp. 5–6, Figure 3 · p. 6, Figure 2

1) Convergence of the scatter matrix MLE:

2) Bias and consistency analysis:

  • For β values 0.2, 0.5, and 0.8, estimated scatter-matrix bias quickly becomes small, while the consistency criterion tends toward a small value as N increases.Source: 2) Bias and consistency analysis · p. 6

C. Unknown shape parameter

  • When M and β are unknown, the proposed algorithm estimates M with a normalized fixed-point step, updates β by Newton–Raphson, and estimates m after the iterations.Source: C. Unknown shape parameter · pp. 6–7

1) Bias and consistency analysis:

The experiments find that scatter-matrix bias and consistency remain favorable across shape parameters, including when the shape parameter is estimated. The real-image study also illustrates MGGD modeling of color-cue dependencies in texture images.

1) Bias and consistency analysis:

2) Shape parameter

  • The MLE variance for β is lower than the method-of-moments variance, and its variance is close to the CRLB across the reported cases.The comparison uses variance versus sample count and β; the reported settings include p = 3 and ρ = 0.8.Source: 2) Shape parameter · p. 7

D. Experiments in a real-world setting

  • For Bark.0000 and Leaves.0008, the estimated MGGD models wavelet-coefficient marginals, and scatter-matrix performance remains similar when β is estimated or fixed.The RGB channels were filtered with a stationary db4 wavelet transform; the study used first-scale, first-orientation coefficients.Source: D. Experiments in a real-world setting · p. 8, Figure 8 · p. 8

VI. CONCLUSION

  • For β ∈(0, 1), the scatter-matrix MLE exists and is unique up to a scalar factor; simulations show its unbiasedness and consistency persist when β is estimated jointly.The fixed-point sequence converges to the unique profile-likelihood maximum from any initial symmetric positive definite matrix.Source: §VI · p. 8

APPENDIX A PROOF OF PROPOSITION IV.1

The proof establishes an interior maximum for the profile-likelihood function and connects that maximizer to a fixed point. The fixed-point relation is invariant to positive rescaling.

APPENDIX A PROOF OF PROPOSITION IV.1

  • If cM^FP is a fixed point of fχ, then every positive scalar multiple of cM^FP is also a fixed point, by homogeneity of degree one.The proof therefore treats the fixed point up to scale.Source: Appendix A · p. 8
  • The profile-likelihood function extends continuously to nonzero singular matrices with value zero there, so its positive maximum on the compact domain lies in the interior.The proof identifies an interior maximizer after establishing continuity and positivity on the domain.Source: Appendix A · p. 9
  • A maximizer of Fχ is stationary, and the proof shows that this stationary point satisfies the fixed-point equation.This is the link from maximizing the function to obtaining a fixed point.Source: Appendix A · p. 9

APPENDIX B PROOF OF PROPOSITION IV.2

The proof establishes order properties for fχ on positive-definite matrices. It also identifies the equality condition for its second property.

APPENDIX B PROOF OF PROPOSITION IV.2

  • When M ≤ Q, inversion reverses the matrix order, yielding property (P1) for fχ.The proof applies the inverse-order relation to each sample vector.Source: Appendix B · p. 10
  • The proof establishes property (P2) using the definition of fχ and normalized sample vectors; equality requires the stated condition for every sample.The function fχ is unchanged when each xi is replaced by ni = xi/∥xi∥.Source: Appendix B · pp. 10–11
  • The equality condition in (P2) holds exactly when M and Q are collinear, as shown by the cited result.The appendix invokes the result from for this characterization.Source: Appendix B · p. 11

APPENDIX C PROOF OF PROPOSITION IV.3

The proof uses strict monotonicity under matrix ordering and hypothesis (H) to establish the stated equality result. It also relates the argument to earlier work for β = 0.

APPENDIX C PROOF OF PROPOSITION IV.3

  • If fχ(Q) = fχ(P) under Q ≥ P, equality forces (P^-1 − Q^-1)xi = 0 for every i; hypothesis (H) then gives Q = P.This is the equality case stated in (C.32).Source: Appendix C · p. 11
  • For Q ≥ P with Q ≠ P, the proof obtains fχ(Q) ≥ fχ(P) with strict inequality, using an index where the ordered matrices differ.The argument starts from the order relation and shows at least one sample term changes strictly.Source: Appendix C · p. 11
  • The proof notes that its result reduces to the one in when β = 0 and reuses that work’s argument for the remaining step.The paper also says the proof is similar to the cited proposition, though its function differs.Source: Appendix C · p. 11
Loading 1302.6498v2…