Source-linked AI summary
SIRENE: Supervised Inference of Regulatory Networks
Fantine Mordelet, Jean-Philippe Vert
TL;DR
SIRENE addresses the difficulty of inferring regulatory networks when negative training examples lack a systematic source. It decomposes network inference into smaller classification problems and is evaluated on a benchmark, while operon-level train/test splitting limits interpretation of performance.
Problem
Negative examples are problematic because there is no systematic source of genes known not to regulate a target gene.
Method
SIRENE decomposes the difficult problem of gene regulatory network inference into smaller problems using organismal data inputs.
Results
SIRENE’s behavior is robust to variations in its parameters around the default values.
Takeaways & Limitations
SIRENE is evaluated on a benchmark of known regulatory networks.
Takeaways & Limitations
Splitting genes from one operon between training and test sets can make predictions appear correct because similar expression profiles reveal operon membership.
Abstract
from arXiv · showhide
Living cells are the product of gene expression programs that involve the regulated transcription of thousands of genes. The elucidation of transcriptional regulatory networks in thus needed to understand the cell's working mechanism, and can for example be useful for the discovery of novel therapeutic targets. Although several methods have been proposed to infer gene regulatory networks from gene expression data, a recent comparison on a large-scale benchmark experiment revealed that most current methods only predict a limited number of known regulations at a reasonable precision level. We propose SIRENE, a new method for the inference of gene regulatory networks from a compendium of expression data. The method decomposes the problem of gene regulatory network inference into a large number of local binary classification problems, that focus on separating target genes from non-targets for each TF. SIRENE is thus conceptually simple and computationally efficient. We test it on a benchmark experiment aimed at predicting regulations in E. coli, and show that it retrieves of the order of 6 times more known regulations than other state-of-the-art inference methods.
2.1 SIRENE
SIRENE infers directed TF–gene regulatory relationships by decomposing network inference into local classification problems for each TF. It combines expression profiles with known regulatory relationships to rank candidate interactions.
- Inputs: SIRENE requires expression profiles for each gene and known TF–gene regulation relationships as inputs.Known relationships provide regulated genes and, when possible, genes known not to be regulated.
- Data limitation: Regulatory databases such as RegulonDB can supply known relationships, but they usually do not contain information about absent regulation.The method therefore discusses how to generate negative examples.
- Local models: For each TF, SIRENE trains a binary classifier that separates known targets from non-targets using gene-characterizing data.The implementation uses an SVM, although other supervised binary-classification algorithms could be used.
- Model rationale: SIRENE assumes genes regulated by the same TF are likely to exhibit similar expression patterns without assuming a direct TF-expression relationship.Expression profiles therefore provide the similarity structure used to characterize candidate targets.
- Ranking: Each TF-specific model assigns scores to previously unused genes, and the scores are combined to rank candidate TF–gene interactions.Higher positive scores indicate greater belief that a gene is regulated by the TF.
- Summary: SIRENE decomposes gene regulatory network inference into many local models that characterize genes regulated by each TF.The directed-network setting requires predicting targets for each given TF rather than inferring undirected relationships.
2.2 SVM
SIRENE uses SVMs as TF-specific binary classifiers and ranks genes by their classification scores. Its negative-example scheme uses held-out subsets to reduce problems caused by false negatives and training-set scoring.
- Scoring: The SVM scoring function ranks genes from most likely class +1 to most likely class −1, using a kernel-defined similarity measure.For expression profiles, the implementation uses a Gaussian radial basis kernel with bandwidth parameter σ.
- Parameters: The Gaussian-kernel SVM uses parameters C and σ; the experiments fix them at C = +∞ and σ = 8 for all models.C = +∞ gives a hard-margin SVM, while σ = 8 is motivated by the scaled expression-profile geometry.
- Parameters: Varying C and σ in preliminary experiments produced no significant performance change, suggesting robustness around the default values.All reported results use the default parameter choice.
- Negative examples: Negative examples are difficult because databases rarely record that a TF does not regulate a gene, so unreported pairs mix true and false negatives.This creates a training-data problem for supervised inference.
- Negative examples: SIRENE splits non-target genes into three subsets and trains each model on two subsets while scoring the held-out subset.Repeating this three times yields scores for all genes without known regulation by the TF.
- Negative examples: Held-out scoring allows genes that resemble positive training examples to receive positive predictions despite possible false negatives in training.The procedure is repeated separately for every TF.
2.4 CLR
The evaluation compares SIRENE with CLR using cross-validation on known regulatory networks. The protocol also controls for operon structure, which can otherwise make predictions artificially easy.
- Evaluation: SIRENE is compared with CLR on a benchmark of known regulatory networks using a cross-validation evaluation protocol.Cross-validation ensures performance is measured on predictions not used during supervised model training.
- CLR: For CLR, mutual-information scores receive an adaptive background correction and are converted into z-scores for ranking.The correction compares each pair’s score with the contextual likelihoods of the TF and target gene.
- Cross-validation: The protocol randomly splits genes into three parts, trains on two, evaluates on the third, and averages prediction qualities across the three folds.Each TF is trained and evaluated within the fold-specific procedure.
- Operon control: Operon membership can make test predictions easy because genes in the same operon often have highly similar expression profiles.A model may recognize operons rather than generalize to genuinely new regulatory contexts.
- Operon control: To simulate prediction for new operons, all genes from one operon are kept together in either the training or test subset.Results are reported for both classical cross-validation and operon-preserving splits.
- CLR: CLR receives the same evaluation protocol, but because it is unsupervised, its training set is not used within each fold.Its final ROC and precision/recall curves are computed over all genes simultaneously.
- Metrics: Predicted TF–gene interactions are ranked by decreasing score, and ROC and precision/recall curves are averaged across folds.ROC measures recall against false-positive rate, whereas precision/recall plots precision against recall.
3 Data
The experiments use normalized E. coli expression profiles together with experimentally confirmed TF–gene regulations and operon annotations from public resources.
- Regulation data: The regulation dataset contains 3293 experimentally confirmed regulations between 154 TFs and 1211 genes from RegulonDB.The data were made publicly available by Faith et al. and extracted from RegulonDB.
- Expression data: The expression compendium spans pH changes, growth phases, antibiotics, heat shock, media differences, oxygen concentrations, and genetic perturbations.These conditions provide the varied experimental context for the gene profiles.
- Operon annotations: RegulonDB supplies 899 known E. coli operons, and genes absent from the database are treated as single-gene operons.This yields 3360 operons across the 4345 genes.
- Operon annotations: Operon information is used to construct the folds for the cross-validation procedure.The resulting splits preserve operon integrity during train/test partitioning.
4 Results
On the E. coli RegulonDB benchmark, SIRENE outperformed CLR and other tested methods, especially when evaluation respected operon structure and when transcription factors had more known targets. It was then used to predict additional regulations, including literature-supported examples, while remaining limited by sparse prior knowledge.
- Benchmark comparison: SIRENE’s ROC and precision-recall curves were significantly above CLR’s, with recall several times larger at every precision level.The precision-recall curve was particularly relevant because true regulations were rare among all possible TF-gene pairs.
- Evaluation protocol: SIRENE performed better when operons were kept intact during cross-validation, whereas classical splitting could produce an evaluation bias.The SIRENE-bias curve used classical 3-fold cross-validation, while the reported analysis restricted itself to the correct protocol.
- Benchmark comparison: At 60% precision, SIRENE predicted 6 times more known regulations than CLR, the best method previously tested on this benchmark.At this precision level, SIRENE achieved 44.5% recall, allowing retrieval of almost half of all known regulations.
- Dependence on prior knowledge: SIRENE’s per-TF AUC tended to increase with the number of known target genes, a trend absent from CLR.This result supports the expectation that supervised inference benefits from more known targets for each transcription factor.
- Limitations: SIRENE cannot predict targets for transcription factors with no known targets, and its performance decreases when few targets are known.Consequently, it cannot be used to discover new transcription factors.