Source-linked AI summary
Feature Screening via Distance Correlation Learning
Runze Li, Wei Zhong, Liping Zhu
TL;DR
Ultrahigh-dimensional data create challenges for existing regularization methods. The paper proposes distance-correlation-based feature screening, establishes its sure screening property, and reports improved performance over SIS in simulations.
Problem
Existing regularization methods may not perform well for ultrahigh-dimensional data collected in diverse scientific fields.
Method
The paper proposes a feature screening procedure using distance correlation and defines independence ranking without specifying a regression model.
Results
The proposed procedure has an established sure screening property, and simulations indicate that DC-SIS significantly outperforms SIS and SIRS.
Takeaways & Limitations
The proposed DC-SIS provides a feature-screening procedure with a sure screening property for ultrahigh-dimensional data.
Takeaways & Limitations
DC-SIS may fail to identify some important predictors, and the choice of d is very important during screening.
Abstract
from arXiv · showhide
This paper is concerned with screening features in ultrahigh dimensional data analysis, which has become increasingly important in diverse scientific fields. We develop a sure independence screening procedure based on the distance correlation (DC-SIS, for short). The DC-SIS can be implemented as easily as the sure independence screening procedure based on the Pearson correlation (SIS, for short) proposed by Fan and Lv (2008). However, the DC-SIS can significantly improve the SIS. Fan and Lv (2008) established the sure screening property for the SIS based on linear models, but the sure screening property is valid for the DC-SIS under more general settings including linear models. Furthermore, the implementation of the DC-SIS does not require model specification (e.g., linear model or generalized linear model) for responses or predictors. This is a very appealing property in ultrahigh dimensional data analysis. Moreover, the DC-SIS can be used directly to screen grouped predictor variables and for multivariate response variables. We establish the sure screening property for the DC-SIS, and conduct simulations to examine its finite sample performance. Numerical comparison indicates that the DC-SIS performs much better than the SIS in various models. We also illustrate the DC-SIS through a real data example.
1. INTRODUCTION
The paper introduces DC-SIS, a distance-correlation-based screening procedure for ultrahigh-dimensional data, motivated by the limitations of existing methods and the need for broader applicability. It establishes sure screening under general settings, avoids regression-model specification, supports grouped and multivariate predictors or responses, and performs strongly in simulations.
- Existing feature-selection methods face computational, statistical-accuracy, and algorithmic-stability challenges in ultrahigh-dimensional data.
- Earlier screening procedures established sure screening or ranking results under particular models, including linear, generalized linear, additive, and multi-index settings.
- The paper proposes DC-SIS, which uses distance correlation and is as easy to implement as SIS while coinciding with SIS for normally distributed responses and predictors.
- DC-SIS directly screens grouped predictors and multivariate responses, addressing settings that the aforementioned screening procedures cannot handle.
- DC-SIS possesses the sure screening property under very general model settings, including linear regression, without requiring a specified regression model.
- Simulation studies report that DC-SIS significantly outperforms SIS and SIRS under many model settings, performs very well for grouped variables, and can advantageously screen multiple responses jointly.
CORRELATION
The paper develops a distance-correlation-based independence screening procedure that is model-free and supports nonlinear relationships, grouped predictors, and multivariate responses. Its sure screening property holds under milder conditions than SIS, while retaining equivalence to Pearson-correlation screening in a normal linear setting.
- Distance correlation: Distance covariance is defined through characteristic functions and yields distance correlation for random vectors with finite first moments.The sample version estimates the component terms by moments and forms sample distance covariances and correlation.
- Distance correlation: Distance correlation equals zero exactly under independence, allowing screening to detect dependence beyond marginal linear association.For normally distributed variables, the distance-correlation ranking is equivalent to marginal Pearson-correlation learning because the relationship is strictly increasing in absolute Pearson correlation.
- DC-SIS procedure: DC-SIS ranks predictors using marginal distance correlation and defines active predictors through functional dependence of the conditional response distribution on each predictor.Inactive predictors are redundant given the active predictor set under the paper’s conditional-independence formulation.
- DC-SIS procedure: The procedure permits arbitrary regression relationships, univariate or multivariate responses, continuous, discrete, or categorical outcomes, and grouped predictors without specifying a model.These features make DC-SIS completely model-free in the stated setting.
- Theoretical properties: Under tail, minimum-signal, and technical conditions, DC-SIS has the sure screening property and can handle NP-dimensional settings.The paper states that this property does not require a linear regression function and compares favorably with the conditions for SIS.
- Theoretical properties: DC-SIS can effectively handle grouped predictors and multivariate responses compared with SIRS, while its assumptions include sub-exponential tails and sufficiently large active-predictor distance correlations.The minimum active marginal distance correlation is required to be at least 2cn^-κ, with 0 ≤ κ < 1/2.
3. NUMERICAL STUDIES
The numerical studies evaluate DC-SIS through simulations involving nonlinear, interaction, heteroscedastic, grouped-predictor, and multivariate-response settings, plus a microarray example. DC-SIS performs similarly to competing methods near linear settings, but substantially better in several nonlinear settings and supports grouped and multivariate screening.
- Simulation design and evaluation: The simulations assess screening using minimum model size S, individual-selection proportion Ps, and all-active-selection proportion Pa across 500 replications.S measures model complexity; Ps and Pa assess whether active predictors are selected.
- Example 1: In model (1.a), DC-SIS, SIS, and SIRS have quite similar performance, indicating robust SIS behavior when the working linear model is close to the true model.Models (1.a)–(1.d) are nonlinear in X1^2; models (1.b) and (1.c) include interactions, while model (1.d) is heteroscedastic.
- Example 1: DC-SIS significantly outperforms SIS and SIRS in models (1.b), (1.c), and (1.d), where SIS and SIRS have little chance to identify important predictors.The difficult settings include interaction terms and heteroscedasticity.
- Example 2: DC-SIS can directly screen grouped predictors, and almost all Ps and Pa values equal 100% when selecting models of sizes based on [n/log n].In Example 2, the grouped variable ex12 is almost perfectly selected across all three model sizes; [n/log n] = 37.
- Example 3: For multivariate responses, active-predictor selection proportions are close to one, supporting the sure screening property and identification of predictors involved in response correlations.The DC-SIS is presented as directly applicable because of the nature of distance correlation.
- Example 4: In the Cardiomyopathy microarray example, DC-SIS ranks Msa.2134.0 and Msa.2877.0 first, with scatterplots indicating nonlinear patterns.Using these two predictors, the reported adjusted R2 is 96.8% and deviance explained is 98.3%, versus 84.5% and 86.6% for generalized correlation ranking.
4. DISCUSSION
The discussion presents DC-SIS as a distance-correlation screening procedure with a sure screening property, while identifying practical tuning and methodological limitations that motivate future work.
- Contribution: The paper proposes a sure independence screening procedure using distance correlation and establishes its sure screening property.
- Empirical evaluation: The proposed procedure’s screening performance was examined through Monte Carlo studies and a real data example.
- Practical implementation: The cutoff d is important in the screening stage, but the paper follows Fan and Lv (2008) to set it.
- Practical implementation: The choice of d remains a practical issue, with existing false-positive-rate selection work described as limited to model-based screening methods.
- Limitations and future work: DC-SIS may fail to identify important predictors marginally independent of the response, motivating iterative procedures.
- Limitations and future work: Empirical studies suggest iterative DC-SIS may address this issue, but its theoretical analysis requires further study.
Appendix A: Some Lemmas
The appendix introduces concentration lemmas for bounded variables and U-statistics, which support the theoretical analysis of the screening procedure.
- The stated exponential inequalities are extracted from results in Serfling (1980).
- Lemma 1 gives an exponential inequality for a bounded random variable Y with mean µ.
- Lemma 2 gives an exponential inequality for a bounded-kernel U-statistic with kernel order m and sample size n.
- The proof uses the symmetry of U-statistics to apply the second lemma’s bound.
- The appendix defines an independent copy of (X_k, y) and the quantities S_k1, S_k2, and S_k3 used in distance-covariance analysis.
Appendix B: Proof of Theorem 1
The proof establishes uniform consistency of the components defining the estimated distance-correlation screening measure, then uses these bounds to prove Theorem 1’s screening result.
- The proof targets uniform consistency of both the numerator and denominator of the estimated screening measure bω_k.
- The tail contribution is controlled by choosing M = cn^γ with 0 < γ < 1/2 − κ.
- The proof separately handles S_k2 and S_k3, using product decompositions and third-order U-statistics to establish uniform bounds.
- The numerator and denominator of bω_k are shown to have the same convergence-rate form under the stated conditions.
- The theorem’s second part follows by showing that failure to retain the active set implies a sufficiently large estimation error for some active predictor.