Source-linked AI summary
Minimum Covariance Determinant and Extensions
Mia Hubert, Michiel Debruyne, Peter J. Rousseeuw
TL;DR
The MCD provides a robust approach to multivariate location and scatter, with computation based on an efficient C-step algorithm. Its extensions address reproducibility, computational cost, and high-dimensional data, while preserving robustness or preventing singular matrices.
Problem
High-dimensional settings make covariance-determinant methods long and less robust, while outlier distances can flag too many observations.
Method
The paper reviews the MCD and its properties, computation through the C-step, breakdown value, and two extensions for deterministic computation and regularized high-dimensional estimation.
Results
The MCD's main properties concerning robustness, efficiency, and equivariance are established, and its resistance to outliers is illustrated on a real data example.
Takeaways & Limitations
The two recent MCD modifications save computing time and enable applications to high-dimensional data.
Takeaways & Limitations
For relatively small n, the outlier-detection procedure often flags too many observations as outlying.
Abstract
from arXiv · showhide
The Minimum Covariance Determinant (MCD) method is a highly robust estimator of multivariate location and scatter, for which a fast algorithm is available. Since estimating the covariance matrix is the cornerstone of many multivariate statistical methods, the MCD is an important building block when developing robust multivariate techniques. It also serves as a convenient and efficient tool for outlier detection. The MCD estimator is reviewed, along with its main properties such as affine equivariance, breakdown value, and influence function. We discuss its computation, and list applications and extensions of the MCD in applied and methodological multivariate statistics. Two recent extensions of the MCD are described. The first one is a fast deterministic algorithm which inherits the robustness of the MCD while being almost affine equivariant. The second is tailored to high-dimensional data, possibly with more dimensions than cases, and incorporates regularization to prevent singular matrices.
INTRODUCTION
The MCD is a highly robust, affine-equivariant estimator of multivariate location and scatter that supports outlier detection and robust multivariate methods. Its practical use expanded with computationally efficient algorithms, including deterministic and high-dimensional regularized extensions.
- MCD provides robust, affine-equivariant estimation of multivariate location and scatter.
- Its resistance to outliers makes MCD useful for outlier detection.
- FastMCD made the MCD computationally efficient and helped drive its broader practical use.
- MCD has been applied in medicine, finance, image analysis, and chemistry.
- The estimator has supported robust principal component analysis, factor analysis, and multiple regression.
- Recent extensions include a deterministic algorithm and a regularized version for high-dimensional data.
DESCRIPTION OF THE MCD ESTIMATOR
The MCD estimates multivariate location and scatter from a subset chosen for minimal covariance determinant, providing robust distances that reduce masking in outlier detection. Its robustness is strongest at lower coverage, while weighting or higher coverage improves efficiency with trade-offs in robustness.
- Outlier detection: In the bivariate wine data, robust distances expose 8 outliers, compared with only three mild outliers from Mahalanobis distances.The classical ellipse is enlarged by outliers, whereas the robust ellipse encloses the regular observations.
- Estimator definition: The MCD uses h observations whose covariance determinant is smallest to estimate location and scatter.The location is the mean of the selected observations, and the scatter estimate is their covariance matrix multiplied by a consistency factor.
- Statistical properties: The MCD has consistency and asymptotic-normality results at elliptical models and broader distribution classes.These results concern the raw location and scatter estimator, the location estimator, and the covariance matrix.
- Efficiency and robustness: At α = 0.5, MCD scatter efficiency is 6% for p = 2 and 20.5% for p = 10 relative to the sample covariance matrix.Using α = 0.75 raises these efficiencies to 26.2% and 45.9%, respectively, but diminishes robustness to outliers.
- Efficiency and robustness: A weighting step raises efficiency at α = 0.5 to 45.5% for p = 2 and 82% for p = 10.The weighted MCD was also applied to the wine example with α = 0.75, producing results similar to smaller α values.
PROPERTIES
The MCD is affine equivariant, highly resistant to contamination, and has bounded influence, with univariate breakdown values attaining affine-equivariant upper bounds. Its equivariance preserves diagnostics under nonsingular reparametrizations, while its influence remains bounded for outliers.
- Affine equivariance: The MCD location and scatter estimators are affine equivariant, so rotations, translations, and rescaling do not affect outlier-detection diagnostics.The property follows from how covariance determinants and sample means transform under affine changes.
- Breakdown value: For continuous distributions, the multivariate MCD reaches breakdown value [(n−p+1)/2], the highest possible for affine-equivariant scatter estimators in general position.The result assumes k(Xn)=p and [(n+p)/2] ≤ h ≤ [(n+p+1)/2].
- Breakdown value: Weighted MCD estimators retain at least the raw MCD breakdown value when the weight function is bounded and becomes zero for large distances.This condition limits the effect of observations assigned negligible weights.
- Influence function: At the standard multivariate normal distribution, the MCD location and off-diagonal scatter influence functions become zero for sufficiently distant outliers, while diagonal effects remain bounded.The influence functions are generally smooth except at the weight-function discontinuity, which can be removed with a smooth weight function.
- Univariate properties: In the univariate case, MCD estimates reduce to the mean and standard deviation of the h-subset with smallest variance, with h = [n/2] + 1 attaining maximal breakdown values.The location and scale breakdown values are [(n + 1)/2]/n and [n/2]/n, respectively.
COMPUTATION
Exact MCD computation is combinatorially difficult, so FastMCD uses C-steps and multiple initial subsets to obtain an efficient approximate solution. The algorithm accelerates convergence but does not guarantee the global minimum and becomes more costly as n grows.
- FastMCD: The exact MCD is very hard to compute because it requires evaluating all subsets of size h, motivating approximate algorithms such as FastMCD.FastMCD replaces exhaustive search with a computationally efficient approximation.
- C-steps: A C-step computes distances from an initial subset, selects the h smallest-distance observations, and produces a covariance determinant no larger than the previous one.Equality holds only when the location and covariance estimates remain unchanged.
- Approximation: C-steps converge in finitely many iterations, but the final determinant is not guaranteed to be the global minimum of the MCD objective.Multiple initial subsets are therefore used, retaining the solution with the lowest determinant.
- FastMCD: FastMCD improves speed by applying only two C-steps to each initial subset, then iterating fully only for the 10 subsets with lowest determinants.The reduction exploits the observation that many globally successful runs already have a small determinant after two C-steps.
- Computational limits: As n grows, FastMCD computation time increases because each C-step requires calculating n distances, although partitioning avoids calculations over the entire dataset.The procedure is described as very fast for small sample sizes.
APPLICATIONS
The MCD has been applied across a broad range of applied fields, including finance, medicine, quality control, geophysics, geochemistry, image analysis, and chemistry.
- Applications: MCD applications span finance and econometrics, medicine, quality control, geophysics, geochemistry, image analysis, and chemistry.The paper notes that this list is far from complete.
MCD-BASED MULTIVARIATE METHODS
MCD-based methods extend robust covariance estimation to regression, principal components, classification, clustering, missing data, resampling, and trimmed estimators. The MCD and C-step ideas also motivate broader robust-method extensions.
- Motivation: Because many multivariate methods rely on covariance estimation, the MCD serves as a building block for robust multivariate techniques.Its trimming procedure also supports robust outlier classification.
- Regression: The MCD analog in regression is Least Trimmed Squares, which fits the h-subset with the smallest sum of squared residuals.FastLTS uses techniques similar to FastMCD, and robust regression outlier maps plot residuals against predictor distances.
- Regression: MCD-based robust distances support linear, categorical-regressor, logistic, multivariate, and errors-in-variables regression methods.In multivariate regression, MCD can be applied directly or to residuals for multivariate LTS estimation.
- Dimension reduction: The MCD scatter matrix supplies eigenvectors for low-dimensional principal components when n > 5p, and supports robust factor analysis and invariant coordinate selection.High-dimensional robust PCA is addressed through ROBPCA and related projection-pursuit methods.
- Dimension reduction: ROBPCA has led to robust principal component regression, robust partial least squares regression, robust PARAFAC, and robust multilevel simultaneous component analysis.These methods are paired with appropriate outlier maps where described.
- Extensions: The C-step idea has been extended to S-estimators, while the LTS subspace estimator generalizes trimmed regression to subspace and orthogonal regression.These extensions broaden the use of trimming and iterative concentration beyond the original MCD setting.
- Extensions: Other extensions include MCD-based Hotelling testing, robust bootstrap and cross-validation, missing-value computation, robust Cronbach alpha, classification, and clustering.The MCD trimming idea also inspired maximum trimmed likelihood, trimmed k-means, least weighted squares, and minimum weighted covariance determinant estimation.
RECENT EXTENSIONS
The paper presents DetMCD, a deterministic and faster alternative to FastMCD, and MRCD, a regularized extension for high-dimensional data. DetMCD is permutation invariant and nearly affine equivariant, while MRCD remains invertible when conventional scatter estimates are singular.
- DetMCD: DetMCD replaces FastMCD’s random starts with deterministic initial estimates, making results permutation invariant and less sensitive to point contamination.It uses the same iteration steps as FastMCD but does not begin from random subsets.
- DetMCD: DetMCD runs faster than FastMCD and can compute estimates across a range of h values and corresponding breakdown values.Only the later C-steps depend on h, enabling efficient computation over multiple choices.
- DetMCD: DetMCD standardizes variables using medians and Qn scale estimates, yielding location and scale equivariance under nonsingular diagonal transformations.Six preliminary scatter estimates are refined through eigenvalue recalibration, distance-based subset selection, C-steps, and a final weighting step.
- DetMCD: Although DetMCD is not affine equivariant, its deviation from affine equivariance is very small.Diagnostics such as the scatter estimate’s condition number can provide additional insight into data structure.
- MRCD: MRCD addresses high-dimensional settings by combining subset covariance with a positive definite target matrix, including cases where n < p.The resulting matrix is always positive definite and therefore invertible, even when the subset covariance is singular.
- MRCD: The MRCD algorithm uses deterministic starts and modified C-steps, and the method simulates well in 1000 dimensions.Regularization prevents the zero eigenvalues that make conventional high-dimensional scatter estimates singular.
CONCLUSIONS
The paper reviews the MCD estimator, its robustness and computational properties, and its applications and generalizations. It highlights two recent modifications that reduce computing time and address high-dimensional data.
- CONCLUSIONS: The paper reviews the MCD estimator for multivariate location and scatter, including its robustness, efficiency, equivariance, computation, applications, and generalizations.A real-data example illustrates its resistance to outliers.
- CONCLUSIONS: The reviewed results position MCD as a building block for robust multivariate statistical methods and outlier detection.This consequence is stated in the paper context rather than the supplied conclusion passages.
- CONCLUSIONS: Two recent MCD modifications save computing time and enable analysis of high-dimensional data.These modifications are the deterministic DetMCD algorithm and the regularized MRCD method.