Source-linked AI summary
Selecting between-sample RNA-Seq normalization methods from the perspective of their assumptions
Ciaran Evans, Johanna Hardin, Daniel Stoebel
TL;DR
Normalization methods rely on assumptions that connect RNA-Seq read counts to meaningful cross-sample expression comparisons, but those assumptions can fail across experiments. The paper examines these assumptions, shows that violations impair normalization and downstream differential-expression analysis, and argues that method selection should match the experiment.
Problem
RNA-Seq normalization lacks consensus, while insufficient attention to method assumptions can produce misleading comparisons and inflated false positives.
Method
The paper evaluates between-sample normalization methods by relating their assumptions to biological scenarios involving total mRNA per cell, expression asymmetry, and differential-expression definitions.
Results
Normalization methods perform as expected when their assumptions hold, whereas violated assumptions can produce incorrect normalization and inflated false positives in downstream testing.
Takeaways & Limitations
Researchers should choose a normalization procedure whose assumptions are appropriate for the biological experiment and whose output supports meaningful comparisons across samples.
Takeaways & Limitations
Global shifts in expression may go undetected without spike-in controls, limiting normalization methods that assume balanced expression.
Abstract
from arXiv · showhide
RNA-Seq is a widely-used method for studying the behavior of genes under different biological conditions. An essential step in an RNA-Seq study is normalization, in which raw data are adjusted to account for factors that prevent direct comparison of expression measures. Errors in normalization can have a significant impact on downstream analysis, such as inflated false positives in differential expression analysis. An under-emphasized feature of normalization is the assumptions upon which the methods rely and how the validity of these assumptions can have a substantial impact on the performance of the methods. In this paper, we explain how assumptions provide the link between raw RNA-Seq read counts and meaningful measures of gene expression. We examine normalization methods from the perspective of their assumptions, as an understanding of methodological assumptions is necessary for choosing methods appropriate for the data at hand. Furthermore, we discuss why normalization methods perform poorly when their assumptions are violated and how this causes problems in subsequent analysis. To analyze a biological experiment, researchers must select a normalization method with assumptions that are met and that produces a meaningful measure of expression for the given experiment.
INTRODUCTION
RNA-Seq read counts require between-sample normalization to support meaningful comparisons, but methods depend on assumptions about expression and sequencing. This paper frames normalization through those assumptions to explain method failures and guide experiment-specific choices.
- RNA-Seq maps sequenced cDNA fragments to genes, using mapped read counts to quantify expression.
- Between-sample normalization accounts for sequencing depth, whereas length and GC-content are within-sample effects.
- The paper organizes normalization methods by assumptions and evaluates why violations affect downstream performance.
- The paper defines true expression as mRNA/cell while noting that experiments may instead target a gene’s proportion of transcriptome mRNA.
- Highly expressed genes can alter mRNA and read shares, creating false differential-expression appearances for non-DE genes.
- Global expression shifts can conceal true differential expression or make up-regulated genes appear down-regulated after conventional normalization.
NORMALIZATION METHODS AND ASSUMPTIONS
Normalization translates relative RNA-Seq read counts into meaningful expression measures through assumptions about total expression, differential-expression symmetry, and gene behavior. The paper uses contrasting examples to show how these assumptions shape interpretation.
- Normalization must make normalized read counts represent true mRNA/cell differences rather than merely relative sequencing shares.
- Figure 1: One highly expressed gene: Figure 1 shows one highly expressed gene changing read shares and making non-DE genes appear differentially expressed.
- Figure 2: Global shift in expression: Figure 2 shows a two-fold global up-regulation that conventional normalization can conceal because relative read distributions remain unchanged.
- Figure 3: Differential expression and (a)symmetry: Differential-expression symmetry is defined by equal numbers of genes up-regulated in each condition, independent of whether total mRNA/cell is equal.
Normalization by Library Size
Library-size normalization removes differences in sequencing depth by scaling counts to sample totals. Its interpretation as absolute expression depends on equal total mRNA/cell across conditions.
- Library-size normalization divides each read count by the total number of reads in its sample.
- The key assumption is equal total mRNA/cell across experimental conditions.
- Total Count, RPKM, and FPKM assume or encode sample-level scaling, with RPKM and FPKM additionally accounting for gene length.
- When total mRNA/cell is equal, normalized counts reflect each gene’s absolute mRNA/cell; otherwise they reflect proportions rather than absolute amounts.
Normalization by Distribution/Testing
Distribution- and testing-based methods use non-DE genes to estimate between-sample normalization, relying on comparable technical effects and assumptions about differential-expression balance. Their procedures differ in how they identify or summarize those genes.
- These methods normalize by comparing sample distributions or by testing to identify a set of non-DE genes.
- Both approaches assume technical effects affect DE and non-DE genes similarly.
- Balanced expression assumes roughly equal numbers of genes are up- and down-regulated across conditions.
- Normalization by distribution: Quantile normalization forces sample distributions to match, while Upper Quartile and Median normalization target specific distribution quantiles.
- Normalization by distribution: TMM trims genes using fold-change and expression criteria before scaling counts by the trimmed mean and library size.
- Normalization by testing: PoissonSeq and DEGES iteratively estimate scaling factors while identifying genes considered non-differentially expressed.
Normalization by Controls
Control-based normalization is needed when other methods’ assumptions are violated, provided suitable controls exist and reflect the technical effects affecting other genes. Negative controls, housekeeping genes, spike-ins, and RUV variants rely on distinct assumptions about control behavior and unwanted variation.
- Controls as anchors: Control-based normalization can recover correct expression estimates when global expression shifts make other normalization assumptions invalid.Negative controls should remain non-differentially expressed, allowing normalization to equalize their normalized counts across conditions.
- Control assumptions: Controls must exist, remain non-DE as expected, and experience technical effects that reflect those affecting non-control genes.
- Control types: Housekeeping-gene normalization assumes housekeeping genes are non-DE and affected by technical effects like differentially expressed genes.
- Control types: Spike-in normalization assumes synthetic controls are unaffected by biological conditions and share technical effects with real genes.
- RUV methods: RUVg, RUVs, and RUVr differ in their control requirements and assumptions about whether unwanted variation correlates with experimental condition.RUVg uses negative controls; RUVs additionally uses negative control samples; RUVr instead requires a known design matrix.
IMPORTANCE OF THE ASSUMPTIONS
Normalization performance depends on whether method assumptions match total mRNA per cell and the symmetry of differential expression. Simulations and downstream testing show that assumption violations distort fold changes and can inflate false discoveries.
- Assumption matching: Library-size normalization works when total mRNA/cell is equivalent across conditions, but differing total mRNA/cell can hide DE genes or misclassify non-DE genes.
- Assumption matching: Distribution- and testing-based normalization tolerate differences in mRNA/cell under symmetry but are affected by increasing asymmetry between up- and down-regulated genes.
- Simulation results: Simulation results confirm that Total Count follows Oracle when total mRNA/cell is equal, whereas DESeq, TMM, and DEGES perform well under symmetry.
- Simulation results: PoissonSeq diverges when the proportion of DE genes becomes too high because its fixed-size normalization gene set necessarily includes DE genes.
- Downstream consequences: When normalization fails, observed fold changes depart from truth and downstream differential-expression testing produces inflated false positives.
Simulation Details
The simulations varied differential-expression proportion, asymmetry, and total mRNA/cell across four experimental settings. Six normalization methods were evaluated using non-DE fold-change MSE and empirical FDR, with Oracle normalization as a benchmark.
- Evaluation design: Average non-DE log fold-change MSE and empirical FDR were computed across varying proportions of DE genes, asymmetry, and relative mRNA/cell.
- Simulation conditions: Four simulation sets crossed asymmetric versus symmetric expression with equal versus different mRNA/cell.
- Methods: Six methods—DEGES, DESeq, Oracle, PoissonSeq, TMM, and Total Count—represented distinct normalization strategies.
- Evaluation design: Empirical FDR was calculated from Benjamini–Hochberg-adjusted p-values as the average false-discovery proportion across 50 repetitions.
- Data generation: Each simulation generated 1000 genes across 10 samples, with five samples per condition and a random subset selected as DE.
- Simulation conditions: Asymmetric settings used 75% versus 25% up-regulation, while symmetric settings used 50% up-regulation in each condition, with specified fold changes for equal or different mRNA/cell.
EVALUATION OF METHODS AND ASSUMPTIONS
Normalization methods should be selected according to the definition of differential expression and the assumptions appropriate for the experiment. Comparisons suggest strong performance for DESeq and TMM in many settings, but global shifts, missing controls, and unreliable housekeeping or spike-in assumptions constrain confidence.
- Assumption violations: Differences in mRNA/cell can make library-size normalization perform poorly, especially when highly expressed differentially expressed genes dominate read counts.One evaluation found 50% of reads concentrated in 5% of genes, while another found 50% in 45 male-fly genes and 186 female-fly genes.
- Method performance: DESeq and TMM generally perform well for detecting differentially expressed genes while controlling false positives.Their validity remains uncertain when their assumptions are substantially violated.
- Literature comparisons: Normalization comparisons are summarized as approximate rankings based on each study’s evaluation criteria, with best-performing methods placed at the top.Some rankings were inferred from authors’ discussions of method strengths and weaknesses.
- External controls: Global expression shifts can violate assumptions of library-size, distribution/testing, and housekeeping-gene normalization, making external controls potentially necessary.Spike-in controls may detect shifts missed by methods lacking controls, although spike-ins introduce assumptions of their own.
- External controls: Spike-in normalization has mixed support because studies report both low alignment error and linear read-count relationships, alongside violations of non-differential and shared-technical-effect assumptions.Reported spike-in read counts also depended partly on the mRNA enrichment protocol.
- Recommendations: Method choice must begin with the definition of differential expression, then match the assumptions required for that definition and experiment.Library-size normalization can be poor for mRNA/cell differential expression but appropriate for mRNA/transcriptome differential expression.
CONCLUSION
Normalization methods rely on assumptions that may be violated across experiments, affecting results and downstream analyses. Choosing a method requires considering which assumptions are reasonable, while external controls may be appropriate when assumption validity is unknown.
- Incorrect normalization can produce downstream errors, including inflated false positives, and no consensus exists on the best method.
- Each normalization procedure relies on assumptions, and violating them can lead to incorrect results.
- When assumption validity is unknown, external controls may be appropriate if they can be trusted.
- Published-data analyses evaluating normalization assumptions are lacking, and global shifts may go undetected without additional information.
The False Discovery Rate
The false discovery rate measures the expected proportion of false discoveries among rejected hypotheses. The paper describes the Benjamini–Hochberg procedure and uses it for false-discovery control in simulations.
- The false discovery rate concerns the proportion of discoveries that are false, represented by V/R when R > 0.
- The Benjamini–Hochberg procedure orders p-values, finds a largest qualifying index, and rejects the p-values through that index.
- The BH procedure controls FDR at a desired level α under its stated framework.
- FDR control is distinct from FWER control: FWER control implies FDR control, but FDR control need not imply FWER control.
- Although more advanced procedures can improve power while maintaining FDR control, BH remains common and is used in this paper’s simulations.
Details on Normalization
The paper details normalization procedures that adjust counts using library size, gene length, distributions, sequencing-depth estimates, trimmed comparisons, or modeled unwanted variation. These procedures differ in their normalization strategies and assumptions.
- Library-size and length normalization: Total Count normalization divides each read count by the total reads in its sample to account for library-size differences.
- Library-size and length normalization: RPKM additionally divides counts by gene length, while FPKM uses cDNA fragments rather than RNA reads.
- Distribution-based normalization: Quantile normalization sorts each column, replaces sorted entries with row means, and restores the original ordering, forcing samples toward a common distribution.
- Distribution-based normalization: Upper Quartile and Median normalization scale counts using the sample’s 75th percentile or median, respectively.
- Size-factor and comparison-based normalization: DESeq estimates sample size factors through a pseudo-reference, while CuffDiff applies related internal and external scales within and across biological conditions.
- Size-factor and comparison-based normalization: TMM trims genes by fold-change and expression level before computing a weighted mean relative to a reference sample.
- Model-based normalization: PoissonSeq models expected counts using sequencing depth, baseline gene expression, and condition association, while RUVg estimates unwanted-variation factors from negative controls.
KEY POINTS
Normalization methods depend on assumptions that translate raw read counts into meaningful expression measures, so method choice must match the biological experiment. Violated assumptions can undermine downstream results, and some global expression shifts require controls.
- Assumptions allow normalization to translate raw read counts into meaningful measures of expression.
- The appropriate normalization method depends on which assumptions are valid for the biological experiment.
- Incorrect normalization can cause downstream problems, including inflated false positives, making results untrustworthy.
- Global shifts in expression can violate conventional normalization assumptions and require controls.
- No normalization method is perfect because every method has cases in which its assumptions are violated.
- Understanding assumptions can help researchers choose the most suitable normalization method for a given experiment.