Source-linked AI summary
Variable selection and regression analysis for graph-structured covariates with an application to genomics
Caiyan Li, Hongzhe Li
TL;DR
The paper addresses variable selection when regression predictors are linked on a graph. It uses graph-constrained regularization with a Laplacian-based smoothness penalty, and reports better variable identification than Lasso and elastic net in its simulations. In the brain-aging gene-expression analysis, the procedure tends to identify linked gene sets that can aid biological interpretation.
Problem
The paper seeks to select relevant variables in regression when explanatory variables are related on a graph, a structure existing approaches do not specifically account for.
Source: §2 · p. 4Method
Grace combines an L1 sparsity penalty with a Laplacian-based graph penalty to incorporate relationships among predictors, with an adaptive form for differing coefficient signs.
Source: §2.1 · pp. 5–6Results
The proposed procedure showed better variable identification than Lasso and elastic net in simulations, while its brain-aging analysis tended to identify linked gene sets unlike Lasso.
Source: §6 · p. 18Takeaways & Limitations
In the brain-aging gene-expression analysis, identifying sets of linked genes often led to better biological interpretation of the selected genes.
Source: §6 · p. 18Grace may perform poorly when linked variables have coefficients with different signs, because those coefficients are not expected to be locally smooth.
Source: §2.1 · p. 6
Abstract
from arXiv · showhide
Graphs and networks are common ways of depicting biological information. In biology, many different biological processes are represented by graphs, such as regulatory networks, metabolic pathways and protein--protein interaction networks. This kind of a priori use of graphs is a useful supplement to the standard numerical data such as microarray gene expression data. In this paper we consider the problem of regression analysis and variable selection when the covariates are linked on a graph. We study a graph-constrained regularization procedure and its theoretical properties for regression analysis to take into account the neighborhood information of the variables measured on a graph. This procedure involves a smoothness penalty on the coefficients that is defined as a quadratic form of the Laplacian matrix associated with the graph. We establish estimation and model selection consistency results and provide estimation bounds for both fixed and diverging numbers of parameters in regression models. We demonstrate by simulations and a real data set that the proposed procedure can lead to better variable selection and prediction than existing methods that ignore the graph information associated with the covariates.
1. Introduction.
The introduction places graph-informed regression among methods for structured covariates and presents a graph-constrained procedure for variable selection. The paper also develops its implementation and theoretical properties.
1. Introduction.
- Earlier structured-covariate methods include group Lasso for grouped predictors, elastic net for highly correlated variables, and fused Lasso for smooth coefficient profiles.The introduction also notes prior consistency and oracle-property results for these methods.Source: §1 · p. 2
- Graph-structured covariates are predictors measured on graph nodes, where links can represent gene-function relationships or physical neighborhood, and edge weights can encode link uncertainty.Unlike fused Lasso, the graph-constrained approach uses prior graph information.Source: §1 · p. 2
- The paper considers graph-constrained regression, presents a cyclical coordinate descent algorithm, extends the procedure to accommodate neighboring coefficients with different signs, and establishes estimation results.The stated theory includes sign consistency, estimator error bounds, and L2 consistency.Source: §1 · p. 3
2. Regression analysis for covariates measured on a graph.
This section sets up regression with graph-measured predictors and introduces graph-constrained estimation, its adaptive sign-aware form, and a coordinate descent implementation.
2. Regression analysis for covariates measured on a graph.
- The model treats measurements at graph nodes as predictors in a fixed-design linear regression, assumes centered predictors and response, and targets selection of relevant variables while accounting for graph relationships.For large p, it assumes sparsity: most true coefficients are exactly zero.Source: §2 · pp. 3–4
2.1. Graph-constrained regularization and variable selection.
- Grace combines sparsity and graph smoothness using a Laplacian-based penalty; its tuning parameters control these components, and special cases recover Lasso or elastic net.The graph variation penalty discourages large coefficient differences across linked nodes, with degree scaling described in the section.Source: §2.1 · pp. 5–6
- Grace may perform poorly when linked variables have coefficients with different signs, because those coefficients are not expected to be locally smooth.The proposed adjustment uses initial coefficient estimates to modify the graph penalty for sign differences.Source: §2.1 · p. 6
2.2. An adaptive graph-constrained regularization.
- The adaptive Grace construction changes the graph Laplacian according to the signs of initial coefficient estimates and retains a positive semi-definite matrix.The section names the resulting estimator adaptive Grace (aGrace).Source: §2.2 · p. 6
2.3. A coordinate descent algorithm.
- A coordinate descent algorithm updates coefficients one at a time; connected predictors incorporate neighboring coefficient estimates, while isolated predictors use the Lasso update.The algorithm computes a solution path over decreasing penalty values and uses cross-validation to select tuning parameters.Source: §2.3 · pp. 6, 8
3. Error bound and model selection consistency for fixed and diverging p.
Grace is analyzed through finite-sample error and consistency results, then sign-consistency guarantees for both fixed and diverging dimensions. The guarantees depend on regularity and graph-constrained conditions.
3. Error bound and model selection consistency for fixed and diverging p.
- The theoretical results cover error bounds, L2 consistency, and model-selection consistency for both fixed and diverging numbers of covariates.The assumptions include conditions on predictor-matrix behavior; the stated conditions hold naturally when rows are i.i.d. with finite second moments.Source: §3 · pp. 8–9
3.1. Error bound and L2-consistency of Grace.
- Under (A1), Grace has a nonasymptotic risk bound for any p and q<p; further conditions yield L2 consistency, high-probability recovery of important variables, and small false-selected coefficients.The consistency result also motivates thresholding small Grace coefficients.Source: §3.1 · p. 9, §3 · p. 9
3.2. Model selection consistency when p is fixed.
- For fixed p, q, and β, under a positive-definite limit for C and condition (A.2), Grace is sign consistent if and only if GC-IC holds.In special cases, GC-IC reduces to Lasso’s irrepresentable condition.Source: §3.2 · p. 10
3.3. Model selection consistency when p diverges.
- For diverging p, q, and n, Theorem 3.3 establishes sign-consistent selection under GC-IC and additional conditions on dimension scaling and tuning parameters.The theorem treats general scalings of p, q, and n.Source: §3.2 · p. 11, §3.3 · p. 11
4. Monte Carlo simulations.
In simulated gene-regulation modules, Grace and aGrace generally improved variable selection over Lasso and Enet in the tested coefficient-sign settings. Prediction comparisons depended on the model and correlation level.
Simulation design
- The simulations used 2,200 variables, with 44 relevant variables drawn from four transcription factors and their regulated genes across 200 regulatory modules.Each model used training, validation, and test sets of 200 samples, with 100 simulation repetitions; Model 2 included neighboring genes with differing coefficient signs.Source: §4 · pp. 11–12
Prediction
- Grace had the smallest prediction errors in Model 1 across correlations, while aGrace had the smallest in Model 2; at very high correlation, Enet was slightly better than Grace in the general comparison.Grace and Enet were otherwise reported to have similar prediction results; when neighboring coefficients had the same signs, aGrace reduced to Grace.Source: §4 · pp. 12–13, Table 1 · p. 13
Variable selection
- Grace had larger ROC areas than Enet and Lasso in Model 1, while aGrace led the four procedures in Model 2; Grace also beat Lasso and Enet for Model 2 at both tested correlations.The figure compares models at correlations 0.2 and 0.9; the ROC curves vary with λ1, with λ2 selected by cross-validation.Source: §4 · pp. 13–14, Figure 1 · p. 14
5. Application to network-based analysis of gene expression data.
The application combines human brain gene-expression data with KEGG network information to identify aging-associated subnetworks. Graph-aware procedures selected more connected genes, and adaptive Grace found small subnetworks missed by Grace and elastic net.
5. Application to network-based analysis of gene expression data.
- The analysis combined human-brain gene-expression data with KEGG regulatory-network information to identify subnetworks associated with brain aging, producing a final network of 1,305 genes and 5,288 edges.Expression data came from postmortem human frontal cortex samples from 30 individuals aged 26–106 years.Source: §5 · p. 15
- Adaptive Grace selected more genes at similar cross-validation errors; Lasso selected the fewest with relatively large errors, while Grace and elastic net were similar.Table 2 reports gene counts, linked KEGG edges, five-fold cross-validation errors, and tuning values.Source: §5 · p. 15, Table 2 · p. 15
- Elastic net and Grace identified brain-aging-associated subnetworks that included FGF genes and receptors, MAPK/MAP2K genes, RASGRF1, and INSR.The passage relates these identified genes to developmental, neural, aging, or energy-metabolism functions.Source: §5 · p. 16
- Adaptive Grace identified small connected gene sets missed by elastic net or standard Grace, including EPHRIN/Eph receptor and part of Jak-Stat signaling.Other identified subnetworks included the T-cell CC chemokine receptor gene and PVRL3–PVRL1.Source: §5 · p. 16
- Grace combines L1 and L2 penalties on degree-scaled coefficient differences across linked variables, inducing sparsity and graph-based smoothness.Simulations indicated better prediction and identification performance than Lasso and elastic net when neighboring variables had similar coefficients.Source: §6 · p. 16
6. Discussion.
The discussion presents graph-constrained regularization as a way to use neighboring-variable information, with performance advantages when neighboring coefficients are similar. It describes broader graph applications and extensions while noting that the adaptive sign-adjusted procedure's theoretical properties remain unknown.
6. Discussion.
- When neighboring coefficients are similar, the proposed procedure can outperform Lasso and elastic net in prediction and identification; if smoothness does not hold, cross-validation may select a small λ2 and yield Lasso-like performance.In brain-aging analysis, the method tends to identify linked gene sets, which can aid biological interpretation.Source: §6 · p. 18
- The methods apply to covariates on general graphs, with image analysis given as an example beyond genomics.The paper also presents Fig. 2 subnetworks identified by elastic net, Grace, and adaptive Grace, plotting only genes linked on the KEGG network.Source: Figure 2 · p. 17, §6 · p. 18
- Potential extensions include generalized linear and censored-survival models, alternative sparse penalties, and methods for directed graphs; the adaptive sign-adjusted procedure's theoretical properties remain unknown.The discussion notes that the sign-adjusted procedure can perform better than the original graph-constrained procedure.Source: §6 · p. 18