Feature selection in omics prediction problems using cat scores and false nondiscovery rate control

1. Introduction.

• Small sample sizes complicate classifier training, while the large number of variables makes suitable feature selection difficult in genomic prediction.Source: §1 · p. 1
• Many omics data sets contain predictor correlations, including correlations among biomarkers and genes within functional groups or pathways.DDA simplifies estimation by setting correlations to zero, and PAM provides multiclass feature selection in this diagonal setting.Source: §1 · p. 2
• Correlation-aware regularized LDA methods such as SCRDA, Clanc, and MLDA lack an efficient feature-selection scheme.The cited passage attributes this to multiple optima in SCRDA's regularization parameters and Clanc's large search space for feature subsets.Source: §1 · pp. 2–3
• The proposed framework combines James–Stein shrinkage with analytically estimated parameters, cat-score gene ranking, and FNDR thresholding.The authors report that FNDR thresholding performs similarly to higher criticism.Source: §1 · p. 3

2.1. Standard formulation.

• Standard LDA models each class with a multivariate normal density and shared covariance, then predicts using the class with the highest posterior probability.The discriminant score is defined as the log posterior, after dropping terms constant across groups.Source: §2.1 · p. 3

2.2. Pooled centroid formulation.

• The pooled-centroid restructuring preserves the original LDA predictions while expressing the predictor through vector weights and Mahalanobis-transformed predictors.The weight components control each variable's contribution to the discriminant score.Source: §2.2 · p. 4

2.3. James–Stein shrinkage rules for learning the LDA predictor.

• Training uses James–Stein-type shrinkage estimators for correlations, variances, and class proportions, with shrinkage analytically chosen by minimizing mean squared error.The group centroids are estimated by empirical means.Source: §2.2 · p. 4, §2.3 · p. 5

3.1. A natural variable selection score for LDA.

• Cat scores generalize the t-score criterion used in diagonal discriminant analysis to LDA with correlated predictors.They are correlation-adjusted t-scores and provide the natural LDA generalization of the regularized t-scores used for feature selection in DDA.Source: §3.1 · p. 6
• The feature summary score Si sums squared group-specific cat scores, retaining information when a feature discriminates across multiple groups.The estimated score distribution is approximately χ2, making it more tractable than the alternative maximum-over-groups score.Source: §3.1 · p. 6

3.2. Feature selection by controlling the false nondiscovery rate.

• FNDR control identifies null features for exclusion, whereas FDR control targets genes confidently identified as differentially expressed.Since local FNDR equals one minus local FDR, the classifier retains features with local FDR below 0.8 in the example thresholding scheme, rather than below 0.2.Source: §3.2 · pp. 6–7
• The resulting classifier retains a superset of the features selected as differentially expressed, with an intermediate buffer zone between acceptance and rejection.The figure describes local false discovery rates as a function of Si and distinguishes acceptance, rejection, and buffer zones.Source: §3.2 · pp. 6–7, Figure 1 · p. 7
• The procedure estimates FDR quantities semiparametrically and assumes enough null features to estimate the null model properly.It identifies null features using local FNDR control, then uses all remaining features for prediction.Source: §3.2 · p. 7

4.1. Two groups.

• For two groups, the feature selection score Si reduces to the squared cat score between the two means.The section also refers readers to Zuber and Strimmer (2009) for comparisons of two-group gene rankings with other test statistics.Source: §4.1 · p. 8
• When predictors are uncorrelated, cat scores reduce to standard t-scores and Si to the sum of the corresponding squared t-scores.The discriminant function then takes the standard diagonal discriminant analysis form.Source: §4.2 · p. 8
• In the uncorrelated case, the pooled-centroid LDA formulation reduces to PAM except for PAM’s shrinkage of the means.Source: §4.2 · p. 8

Remark A

• Splitting the inverse correlation equally across the weight and predictor terms makes selection operate on centered, scaled, decorrelated predictors.The authors note that alternative decompositions can make the uncorrelated-case selection criterion depart from t-scores.Source: §5 · p. 8

Remark B

• Summing squared cat scores within a feature set yields Hotelling’s T2, supporting pathway-level grouping.Source: §5 · p. 9

Remark C

• The authors attribute poor prediction from FDR feature selection to naive FDR use that eliminates too many predictors; FNDR control yields more efficient prediction rules.Source: §5 · p. 9

Remark D

• The James–Stein shrinkage estimator permits efficient computation of the inverse square root of the correlation matrix.Source: §5 · p. 9

Remark E

• A cube-root transformation is used to normalize the χ2 null model so standard FDR software can be applied.The null model has unknown parameters.Source: §5 · p. 9

6.1. Singh et al. (2002) gene expression data.

• The Singh analysis used 6033 genes from 102 patients, and prediction error was estimated with balanced 10-fold cross-validation repeated 20 times.Each of the 200 splits retrained the rule and recomputed feature rankings and FDR statistics, incorporating selection into the error estimate.Source: §6.1 · pp. 9, 11
• 0.0640 prediction error with 166 FNDR-selected features was lower than 0.1682 with 53 naive-FDR features, 0.3327 with all 6033, 0.0859 for PAM, and 0.092 for Ebay.Using cat scores, the LDA FNDR rule selected 131 predictors and achieved the smallest overall prediction error on the Singh data.Source: §6.1 · p. 11

6.2. Performance for multiclass reference data sets.

• On the lymphoma data, FNDR selected 162 nonnull features despite no differentially expressed genes when correlation was ignored; PAM did not identify a compact predictive set.On the brain data, the largest and most difficult set, FNDR yielded compact predictors while PAM failed to find a stable feature set; LDA outperformed DDA on both data sets.Source: §6.2 · p. 12
• Simple FDR cutoffs can leave weak, sparse predictors unselected when none has a sufficiently small false discovery rate, as in the lymphoma example.The tested multiclass data included lymphoma, SRBCT, and brain cancer; all methods performed equally well on the easily classified Khan data.Source: §6.3 · p. 12, §6.2 · p. 12

6.3. Comparison with “higher criticism” feature selection.

• Across the higher-criticism comparison, cat-score LDA had smaller prediction error than t-score DDA, while FNDR and higher criticism achieved equal-level performance.Higher criticism selected, on average, fewer features than FNDR but more than FDR-based selection, typically placing its threshold in the buffer zone.Source: §6.3 · p. 13

7.1. Shrinkage discriminant analysis and feature selection.

• The paper combines James–Stein shrinkage for classifier training, cat-score feature ranking, and FNDR-based feature selection.Shrinkage improves parameter estimates rather than serving as a model-selection procedure.Source: §7.1 · p. 13
• Cat scores provide a multiclass LDA ranking for correlated predictors, and their ordering can differ substantially from t-score rankings.The paper reports that cat-score selection is more effective on high-dimensional data and that FNDR control matches higher criticism in predictive accuracy, whereas FDR-based selection leads to inferior prediction rules.Source: §7.1 · pp. 13–14
• For extremely high-dimensional data, where correlation estimation is very difficult, the authors recommend shrinkage DDA with FNDR or higher-criticism selection.Unlike PAM, this procedure involves no randomization, so the prediction rule and selected-feature count are stable.Source: §7.2 · p. 14

7.2. Recommendations.

• For other settings, the authors recommend full shrinkage LDA with cat-score selection, which costs more computation than DDA but can improve accuracy.Relative to DDA, accounting for correlation typically gives more compact feature sets, improved prediction error, or both; the procedure avoids resampling used by competing full-LDA approaches.Source: §7.2 · p. 14

• The small-n, large-p LDA predictor and cat score rely on three different James–Stein-type estimators.Source: Appendix A · p. 14
• The correlation matrix is estimated by shrinking empirical correlations rij toward zero.The cited approach is Sch3fer and Strimmer (2005).Source: Appendix A · p. 14
• Variances are estimated by shrinking empirical estimates vi toward their median.The cited approach is Opgen-Rhein and Strimmer (2007).Source: Appendix A · pp. 14–15
• Class frequencies are estimated following Hausser and Strimmer (2009).Source: Appendix A · p. 15

• NSC/PAM is a DDA method that ranks genes by equation (3.3) and selects features by soft-thresholding with cross-validated prediction error as its criterion.It does not account for gene-wise correlations.Source: Appendix B · p. 15
• Ebay is also DDA-based and links prediction error with false discovery rates through an empirical Bayes model.The passage describes it as similar to PAM but computationally and statistically more efficient, with correlation corrections for prediction errors.Source: Ebay · p. 15
• Clanc and MLDA both use James–Stein shrinkage for pooled covariance estimation, but MLDA uses t-scores while Clanc searches greedily for feature subsets.Source: Clanc and MLDA · p. 15
• SCRDA uses ridge covariance estimation and PAM-like soft thresholding, with tuning and feature selection parameters chosen together by cross-validation.The cited discussion identifies computational expense and difficulty finding unique parameters as its main issues.Source: SCRDA · p. 16

APPENDIX C: COMPUTER IMPLEMENTATION

• The R package sda implements the proposed DDA and LDA procedures and their associated FNDR and higher criticism variable selection.It is freely available under the GNU General Public License version 3 or later from CRAN.Source: Appendix C · p. 16