Source-linked AI summary
Robustly detecting differential expression in RNA sequencing data using observation weights
Xiaobei Zhou, Helen Lindsay, Mark D. Robinson
TL;DR
Count-based RNA-seq differential-expression methods need to balance power with protection against outliers, especially when replication is limited. The paper studies this robustness problem and introduces observation weights that can be integrated into existing frameworks. Overall, the robust method is reported to resist introduced outliers with minimal loss of power, while an extensible simulation system supports comparisons among methods.
Problem
Count-based differential-expression methods must balance statistical power, false-discovery control, and outlier protection, while existing approaches differ in robustness and replication is often limited.
Method
The paper assigns lower weights to observations with large model residuals and uses those weights in both regression and dispersion estimation within existing differential-expression frameworks.
Results
The observation-weight robust method achieves resistance to introduced outliers with minimal loss of power, while outliers can otherwise have global effects on differential analyses.
Takeaways & Limitations
Smoothly downweighting outlying observations appears preferable to globally increasing variance or removing features entirely from downstream testing.
Takeaways & Limitations
The current edgeR framework does not always achieve its false-discovery rate target, and the source of this liberality remains beyond the study’s scope.
Abstract
from arXiv · showhide
A popular approach for comparing gene expression levels between (replicated) conditions of RNA sequencing data relies on counting reads that map to features of interest. Within such count-based methods, many flexible and advanced statistical approaches now exist and offer the ability to adjust for covariates (e.g., batch effects). Often, these methods include some sort of (sharing of information) across features to improve inferences in small samples. It is important to achieve an appropriate tradeoff between statistical power and protection against outliers. Here, we study the robustness of existing approaches for count-based differential expression analysis and propose a new strategy based on observation weights that can be used within existing frameworks. The results suggest that outliers can have a global effect on differential analyses. We demonstrate the effectiveness of our new approach with real data and simulated data that reflects properties of real datasets (e.g., dispersion-mean trend) and develop an extensible framework for comprehensive testing of current and future methods. In addition, we explore the origin of such outliers, in some cases highlighting additional biological or technical factors within the experiment. Further details can be downloaded from the project website: http://imlspenticton.uzh.ch/robinson_lab/edgeR_robust/
1 Introduction
Count-based RNA-seq differential-expression methods must balance power, false-discovery control, and robustness when biological replicates are limited. This study examines outlier sensitivity and proposes observation weights to reduce extreme observations’ influence within existing frameworks.
- Motivation: RNA-seq count-based differential-expression analysis supports comparisons across experimental conditions and can accommodate complex designs such as batch effects.Generalized linear models broaden the designs that can be analyzed beyond simple two-group comparisons.
- Motivation: Limited biological replication makes information sharing across features important for improving differential-expression inference.The paper frames this sharing as part of the tradeoff among statistical power, false-discovery control, and outlier protection.
- Existing methods: DESeq is described as conservative but robust, whereas edgeR can be sensitive to outliers when dispersion estimates are strongly smoothed toward the abundance trend.The same comparison states that edgeR should be more powerful without extreme observations.
- Study scope: The study provides an extensible framework for evaluating tradeoffs among differential-expression methods and investigating biological or technical factors associated with outliers.The framework is designed to support comprehensive testing of current and future approaches.
- Existing methods: A single extreme value can substantially affect regression-parameter estimates in generalized linear-model analyses.The introduction identifies this lack of robustness as a problem for count-based differential-expression inference.
- Proposed approach: The proposed strategy attaches observation weights to downweight outliers and applies them to both regression and dispersion estimation.The approach builds on weighted likelihood ideas and is intended to retain approximate chi-squared likelihood-ratio testing under the null.
2 MATERIALS AND METHODS
The paper formulates RNA-seq counts with a negative binomial GLM and estimates regression and dispersion parameters using likelihood-based procedures that share information across features.
- Standard negative binomial GLM: RNA-seq feature counts are modeled with a negative binomial distribution whose variance combines the mean and dispersion.The dispersion represents the squared biological coefficient of variation.
- Standard negative binomial GLM: The GLM links each feature’s mean count to covariates through a logarithmic link, incorporating experimental conditions, batch effects, and effective library size.Regression coefficients encode effects of interest for differential expression inference.
- Standard negative binomial GLM: Regression coefficients are estimated by maximum likelihood using an iteratively reweighted least-squares algorithm.The score uses residual-like quantities adjusted by the fitted mean and dispersion.
- Dispersion estimation: Adjusted profile likelihood estimates feature-wise dispersion while penalizing regression-parameter estimation.The adjustment uses the Fisher information matrix.
- Dispersion estimation: Dispersion moderation combines individual and shared likelihood information, with the shared component drawn from features having similar average expression.The weighting parameter α controls the contribution of the shared likelihood, while trended dispersion captures the dispersion–mean relationship.
2.3 A robust negative binomial GLM
The robust extension assigns lower weights to observations with large Pearson residuals and reuses those weights when estimating both regression coefficients and dispersion.
- Robust weighting: Observations that deviate strongly from the fitted negative binomial GLM receive lower weights in subsequent estimation iterations.Pearson residuals from the current fit are transformed into observation weights.
- Robust weighting: The same observation weights are incorporated into adjusted profile likelihood dispersion estimation before the regression parameters are re-estimated.Thus, outlier influence is dampened in both dispersion and regression estimates.
- Robust weighting: The Huber weighting function downweights residuals exceeding a tuning threshold while retaining ordinary weighting for less extreme observations.The tuning constant is usually set to 1.345 for approximately 95% efficiency under normality.
- Robust weighting: Observation weights enter the iteratively reweighted least-squares fit through a diagonal matrix that modifies the working-weight information matrix.This changes the Fisher information used for coefficient estimation.
- Implementation: For edgeR users, implementing the robust method requires only a small change to the standard analysis pipeline.
2.4 A simulation framework with parameters based on the joint distribution of mean and dispersion estimates from RNA-seq data
The simulation framework derives negative binomial parameters from real RNA-seq datasets, preserves their dispersion–mean structure, and introduces configurable differential-expression and outlier scenarios.
- Simulation parameters: The framework varies feature number, experimental groups, differential-expression proportion, expression fold difference, and the proportion of upregulated features.
- Simulation parameters: It also varies the source dataset, the proportion of outliers, and the mechanism used to generate them.
- Simulation framework: The simulations generate negative binomial parameters from joint distributions of real-data mean and dispersion estimates.This preserves dataset-specific dispersion–mean trends after filtering extreme dispersions and low means.
- Outlier scenarios: Outliers are introduced either by multiplying counts by random factors between 2 and 10 or by applying alternative single- or random-observation mechanisms.The single mechanism adds an outlier to a selected gene, whereas the random mechanism allows each observation to become an outlier probabilistically.
- Simulation scope: The simulations exclude zero-inflated negative binomial distributions because they did not explain all observed dispersion–mean relationships in the tested datasets.
2.5 Methods compared
The study compares edgeR-robust with a broad set of RNA-seq differential-expression methods using an extensible R-based evaluation framework.
- Methods compared: The comparison includes edgeR, edgeR-robust, limma-voom, multiple DESeq variants, DESeq2, baySeq, SAMseq, EBSeq, and ShrinkBayes.
- Methods compared: The evaluated methods differ substantially in dispersion estimation, including trended moderation, observation weighting, feature-wise safeguards, and other model-based procedures.edgeR-robust extends edgeR’s trended dispersion moderation with observation weights, while DESeq uses the maximum of trend-based and feature-wise estimates.
- Evaluation framework: The framework supports arbitrary additional R-based methods when wrappers provide count tables, grouping variables, and P-value outputs.
- Methods compared: Bayesian methods in the comparison include baySeq, EBSeq, and ShrinkBayes, with baySeq using an empirical-Bayes negative binomial model and ShrinkBayes using a zero-inflated negative binomial model.
2.6 Comparison metrics
The study evaluates differential-expression methods using standard metrics that assess discrimination between differentially expressed and non-differentially expressed features and control of false discoveries.
- False discovery procedures gauge the expected proportion of incorrectly rejected null hypotheses.
- ROC, partial ROC, and power curves evaluate how well methods distinguish differentially expressed from non-differentially expressed features.
2.7 An open graphical tool and R code for re-analysis: evaluating DE analysis methods
The authors provide a standardized, extensible system for simulation-based comparison of differential-expression methods, addressing the repetitive and time-limited nature of conventional method comparisons.
- Method comparisons are often snapshots that require developers to repeatedly reimplement similar metrics, plots, and simulation models.
- The evaluation framework separates simulation, software-package evaluation, and metric computation into extensible modules.
- A web-based Shiny app lets users inspect results across many conditions and select subsets of methods and simulation settings.
2.8 Functional category analysis for outliers
Functional-category analysis is applied to downweighted genes on a per-sample basis to investigate biological or technical factors associated with outliers.
- Genes with observation weights below 1 are analyzed using hypergeometric-based functional-category tests for each sample.
- The analysis seeks possible biological or technical factors affecting subsets of genes in particular experimental units.
3 RESULTS
The results show that outliers can affect differential-expression analyses globally, while observation weighting reduces their influence and generally preserves power. Simulations, real datasets, and functional analyses support robustness benefits and help investigate possible outlier sources.
- Iterative robust reweighting changes outlier-associated expression and dispersion estimates and produces a global drop in the dispersion-mean trend.
- Strong differential-expression evidence remains for miR-133b after downweighting its outlier observations.
- Simulations use negative-binomial parameters sampled from Pickrell data, 10,000 features, 5-versus-5 groups, 10% differential expression, and 10% outliers.
- Introducing outliers increases false positives or reduces true positives at the same false-positive rate, while edgeR-robust shows advantages over edgeR in affected settings.
- All methods lose power after outliers are introduced, but edgeR-robust has the smallest drop; DESeq becomes very conservative under its dispersion policy.
- edgeR-robust leads when outliers reduce differential expression, while edgeR benefits artificially when outliers reinforce the higher-expression group.
- Across broader simulation settings, edgeR-robust retains advantages with outliers and only a minor power decrease without them.
- Downweighted observations correlate with GC content in one dataset, while overrepresented extracellular-matrix categories identify a signal sample in another.
4 DISCUSSION
The study develops a robust observation-weight strategy to reduce outlier influence while preserving differential-expression power, and evaluates it across simulations and real datasets. The authors also provide an extensible evaluation framework and identify technical or biological factors associated with outliers.
- Observation weights smoothly downweight outliers when there is evidence of departure from the model, balancing resistance against statistical power.The approach adapts residual and weighting calculations to negative-binomial GLM estimation with flexible dispersion estimation.
- Outlier investigation can reveal technical or biological factors that help explain unusual observations.The study explores potential origins of outliers in real datasets alongside its robustness analyses.
- The precise degree of dispersion moderation remains an open dataset-specific choice because moderation affects inference performance.The paper notes a tradeoff between moderation toward the dispersion-mean trend and average inference performance.
- The simulation framework samples mean and dispersion parameters from real data and supports broad, extensible method evaluations.New methods can be compared by supplying wrapper functions, and simulation settings can be extended beyond those used in the report.
- The observation-weight method maintains high power with minimal loss while resisting introduced outliers.The authors report that hard outlier policies can instead increase variance globally or exclude differentially expressed features.
- The current edgeR framework does not always achieve its false discovery rate target.The source of this liberality remains beyond the study’s scope, although small-sample asymptotic approximations are suggested as a possible improvement.
5 CONCLUSION
The paper presents observation weighting for robust count-based differential-expression analysis and an extensible simulation system. The method aims to resist outliers while maintaining high power, and the system supports rapid evaluation of methods and settings.
- The method dampens outlier effects in count-based differential-expression analyses while maintaining high power.
- The reweighting idea can be adopted beyond the provided edgeR Bioconductor implementation.
- The simulation system uses existing datasets to evaluate new methods or alternative package settings across broad conditions.An interactive web-based Shiny app displays results across simulation settings.