Source-linked AI summary
Comment on "Detecting Novel Associations In Large Data Sets" by Reshef Et Al, Science Dec 16, 2011
Noah Simon, Robert Tibshirani
TL;DR
The commentary questions whether MIC’s equitability is useful when its statistical power is deficient. It argues that distance correlation is more powerful, simple, and easy to compute.
Problem
MIC’s equitability is questioned because low power would limit its usefulness.
Source: p. 1Method
The commentary proposes distance correlation as a more powerful technique for general use.
Source: p. 1Results
MIC is characterized as having serious power deficiencies that could produce too many false positives in large-scale exploratory analysis.
Source: p. 1Takeaways & Limitations
Distance correlation should be considered for general use because it is described as powerful, simple, and easy to compute.
Source: p. 1The dependency set considered is not exhaustive.
Source: p. 1
Abstract
from arXiv · showhide
The proposal of Reshef et al. (2011) is an interesting new approach for discovering non-linear dependencies among pairs of measurements in exploratory data mining. However, it has a potentially serious drawback. The authors laud the fact that MIC has no preference for some alternatives over others, but as the authors know, there is no free lunch in Statistics: tests which strive to have high power against all alternatives can have low power in many important situations. To investigate this, we ran simulations to compare the power of MIC to that of standard Pearson correlation and distance correlation (dcor). We simulated pairs of variables with different relationships (most of which were considered by the Reshef et. al.), but with varying levels of noise added. To determine proper cutoffs for testing the independence hypothesis, we simulated independent data with the appropriate marginals. As one can see from the Figure, MIC has lower power than dcor, in every case except the somewhat pathological high-frequency sine wave. MIC is sometimes less powerful than Pearson correlation as well, the linear case being particularly worrisome.