Source-linked AI summary
TIGRESS: Trustful Inference of Gene REgulation using Stability Selection
Anne-Claire Haury, Fantine Mordelet, Paola Vera-Licona, Jean-Philippe Vert
TL;DR
Inferring gene regulatory networks from expression data is difficult because the full set of regulations is large and existing methods have limited accuracy. The paper introduces TIGRESS, combining LARS feature selection with stability selection and a new scoring function; it ranks among the top DREAM5 methods, while performance remains weaker on in vivo networks.
Problem
Systematically inferring complete gene regulatory networks from expression data remains difficult because the number of potential regulations is large and existing methods have limited accuracy.
Method
TIGRESS formulates GRN inference as feature selection, combining linear LARS regression with stability selection and a new area-under-the-stability-curve scoring method.
Results
TIGRESS ranked in the top three GRN inference methods at the 2010 DREAM5 challenge without parameter tuning and matched GENIE3 on the in silico network when optimally tuned.
Takeaways & Limitations
Feature selection techniques can support GRN inference, although TIGRESS performs less well than GENIE3 on in vivo networks.
Takeaways & Limitations
Regression-based methods including TIGRESS and GENIE3 appear to reach a performance limit on in vivo networks, with E. coli results around 64% AUROC and 8% AUPR.
Abstract
from arXiv · showhide
Inferring the structure of gene regulatory networks (GRN) from gene expression data has many applications, from the elucidation of complex biological processes to the identification of potential drug targets. It is however a notoriously difficult problem, for which the many existing methods reach limited accuracy. In this paper, we formulate GRN inference as a sparse regression problem and investigate the performance of a popular feature selection method, least angle regression (LARS) combined with stability selection. We introduce a novel, robust and accurate scoring technique for stability selection, which improves the performance of feature selection with LARS. The resulting method, which we call TIGRESS (Trustful Inference of Gene REgulation using Stability Selection), was ranked among the top methods in the DREAM5 gene network reconstruction challenge. We investigate in depth the influence of the various parameters of the method and show that a fine parameter tuning can lead to significant improvements and state-of-the-art performance for GRN inference. TIGRESS reaches state-of-the-art performance on benchmark data. This study confirms the potential of feature selection techniques for GRN inference. Code and data are available on http://cbio.ensmp.fr/~ahaury. Running TIGRESS online is possible on GenePattern: http://www.broadinstitute.org/cancer/software/genepattern/.
1 Background
Gene regulatory network inference seeks to recover transcription factor–target gene interactions from genome-wide expression data, but experimentally characterizing complete networks remains difficult. Regression and feature-selection approaches have shown strong benchmark performance, motivating TIGRESS’s investigation of LARS combined with stability selection.
- Complete experimental characterization of gene regulatory networks remains difficult because of the large number of potential regulations.
- Genome-wide expression measurements enable reverse engineering of transcription factor–target gene interactions across experimental conditions.Many approaches have been proposed to infer regulatory interactions from collections of gene expression data.
- Recent benchmarks identify regression and feature selection as effective formulations for selecting transcription factors predictive of each target gene’s expression.GENIE3 exemplifies this approach by using random forests to identify predictive transcription factors.
- TIGRESS investigates LARS combined with stability selection for gene regulatory network inference and introduces a new scoring technique for that procedure.The method was ranked among the top approaches in the DREAM5 challenge, while parameter tuning produced further performance improvements.
2 Methods
The method formulates GRN inference as target-gene-specific sparse regression and ranks candidate TF–TG regulations using LARS with stability selection. Its area-based score integrates selection frequencies across the LARS path to reduce sensitivity to the path-length parameter.
- GRN inference formulation: GRN inference identifies a subset of candidate regulations E by scoring and thresholding each TF–TG pair, producing a global ranking of candidate regulations.The threshold δ controls how many regulations are predicted, while the score determines their ranking.
- GRN inference formulation: For each target gene, the method predicts expression from candidate TF expression levels but focuses on selecting a small sufficient TF subset rather than estimating the full regression function.Each candidate TF receives a score reflecting how likely it is to participate in the target-gene regression model.
- LARS and stability selection: LARS iteratively adds TFs to a linear model, efficiently producing a ranked list of TFs after L steps without fully re-optimizing the model at every addition.The procedure is closely related to forward stage-wise regression and the Lasso.
- LARS and stability selection: Stability selection repeatedly applies LARS to randomly perturbed expression data, using selection frequencies to reduce sensitivity to correlated features and provide scores aggregatable across target genes.The perturbations split experiments into halves and randomly down-weight candidate-TF expression levels before LARS is run.
- Scoring stability selection: The original stability score uses selection frequency among the top L variables, whereas the proposed area score sums frequency information across the full LARS path up to L.The area score rewards TFs that are frequently selected at small ranks and is intended to be less sensitive to the choice of L.
- Evaluation: For DREAM5 datasets, the authors compute AUROC and AUPR P-values by comparing observed scores with randomly drawn prediction lists.The random lists preserve each predicted-list row position when drawing edges.
3 Data
TIGRESS is evaluated on four benchmark datasets combining simulated or experimentally measured expression data with known transcription factors and verified regulatory interactions. The benchmarks include three DREAM5 datasets and an additional E. coli dataset, with gene-wise standardization as preprocessing.
- TIGRESS and other methods are evaluated on four benchmark datasets with expression data, known TFs, and verified TF-TG regulations.The expression data span simulated and experimentally measured conditions involving genetic, drug, and environmental perturbations.
- Three benchmarks come from DREAM5: one simulated network and two real datasets from E. coli and S. cerevisiae.DREAM5 Network 2 is excluded because it lacks verified TF-TG interactions.
- The additional E. coli benchmark contains 907 experiments, 4297 genes, and 3812 verified interactions among 1525 genes.Expression data come from M3D, while the gold standard comes from RegulonDB.
- Expression levels are mean-centered and scaled to unit variance for each gene within each compendium.
4 Results
TIGRESS ranked third overall in DREAM5, while parameter studies showed that area scoring and careful tuning improve performance on Network 1. Performance gains did not transfer consistently to larger in vivo networks, and error analysis identified biologically plausible false-positive patterns.
- DREAM5 challenge results: TIGRESS ranked third overall in DREAM5, with particularly strong performance on the in silico network and higher AUPR than GENIE3.The submitted version used fixed, non-optimized parameters and the original scoring method.
- Influence of TIGRESS parameters: The area scoring method consistently outperformed the original score and was less sensitive to α and L across Network 1 parameter settings.With area scoring, α between 0.2 and 0.8 and L below 10 yielded an overall score of at least 90, whereas the original score could fall to 60.
- Influence of TIGRESS parameters: The best overall performance in both scoring settings used α = 0.4 and L = 2, while increasing resampling runs improved performance until an apparent asymptote above R = 5,000.Figure 4 fixed L = 2 and α = 0.4 when varying R.
- Comparison with other methods: On Network 1, TIGRESS outperformed all methods in AUPR and all methods except GENIE in AUROC; its top predictions contained more true edges than other methods.The ROC curve evaluates the full ranked list, whereas Precision/Recall highlights the top of the prediction list.
- In vivo networks results: On DREAM5 Networks 3 and 4, tuned TIGRESS did not outperform Naive TIGRESS, and performance remained weak despite larger L appearing preferable for these larger networks.The networks contained 4,511 and 5,950 nodes, and the absolute improvement was not dramatic.
- Error analysis: Most recovered false positives were distance-2 edges, often corresponding to sibling relationships and potentially spurious feed-forward loops.The analysis characterized many top-list errors as sensible guesses involving nodes separated by one intermediate node.
5 Discussion
TIGRESS combines LARS with stability selection and improves scoring robustness through an area-based stability score. It performs similarly to GENIE3 in silico but less well in vivo, where linearity and network-ground-truth issues remain concerns.
- TIGRESS ranked among the top three GRN inference methods at DREAM5 without parameter tuning.
- The area-under-the-stability-curve score improves performance and reduces sensitivity to α and L compared with the original score.The area score uses the full distribution of feature ranks, whereas the original formulation does not.
- TIGRESS matches GENIE3 on the in silico network but performs worse on in vivo networks.GENIE3 uses nonlinear tree-based feature selection, whereas TIGRESS uses LARS and assumes linear relationships between target genes and transcription factors.
- TIGRESS can spuriously infer feed-forward loops when sibling genes are mistaken for regulatory relationships in the E. coli network.The authors suggest operons and nonlinear switch-like regulation as possible explanations, while noting that incomplete gold standards leave the predictions' correctness unresolved.
- In vivo GRN inference remains difficult: E. coli performance levels off near 64% AUROC and 8% AUPR across methods.The authors suggest that surpassing this apparent limit may require changes such as adding supervision to the learning process.