Source-linked AI summary
Removal of Batch Effects using Distribution-Matching Residual Networks
Uri Shaham, Kelly P. Stanton, Jun Zhao, Huamin Li, Khadir Raddassi, Ruth Montgomery, Yuval Kluger
TL;DR
Systematic instrument-related batch effects can distort analyses of technologies such as CyTOF and single-cell RNA-seq. The paper uses a residual network trained with Maximum Mean Discrepancy to align replicate distributions, and reports effective attenuation of batch effects across both data types.
Problem
Systematic measurement errors in technologies such as CyTOF and single-cell RNA-seq can affect statistical analysis when batch effects are not calibrated.
Method
A residual network is trained to minimize Maximum Mean Discrepancy between source and target replicate distributions measured in different batches.
Results
The approach successfully removes significant portions of batch effects in CyTOF and single-cell RNA-seq and outperforms other popular calibration approaches.
Takeaways & Limitations
Residual networks preserve biological properties during calibration, while networks without shortcut connections might distort them.
Takeaways & Limitations
A two-sample test still rejects equality between the calibrated source and target distributions.
Abstract
from arXiv · showhide
Sources of variability in experimentally derived data include measurement error in addition to the physical phenomena of interest. This measurement error is a combination of systematic components, originating from the measuring instrument, and random measurement errors. Several novel biological technologies, such as mass cytometry and single-cell RNA-seq, are plagued with systematic errors that may severely affect statistical analysis if the data is not properly calibrated. We propose a novel deep learning approach for removing systematic batch effects. Our method is based on a residual network, trained to minimize the Maximum Mean Discrepancy (MMD) between the multivariate distributions of two replicates, measured in different batches. We apply our method to mass cytometry and single-cell RNA-seq datasets, and demonstrate that it effectively attenuates batch effects.
1 Introduction
The paper addresses systematic batch effects that make replicate biological measurements differ for irrelevant instrument-related reasons. It proposes MMD-ResNet calibration and evaluates it on CyTOF and single-cell RNA-seq data.
- Batch effects are distributional differences between replicate measurements caused by irrelevant instrument-related factors.They introduce systematic error that can produce spurious analyses or obscure the signal of interest.
- CyTOF replicate specimens measured in different batches can differ noticeably in multivariate protein space, requiring calibration for valid analysis.
- MMD-ResNet learns a map that calibrates a source distribution toward a target distribution when their difference is moderate and near the identity.The source and target are designated from replicate samples measured in different batches.
- The approach removes significant portions of batch effects in CyTOF and single-cell RNA-seq and outperforms other popular calibration approaches.The authors report that comparable CyTOF performance had not previously been reported to their knowledge.
- Shortcut connections are crucial because networks without them might distort biological properties, while indirect calibration can connect replicates through other specimens.
2 Preliminaries
The preliminaries define MMD as a distributional distance and describe residual networks as identity-preserving architectures. These properties motivate using ResNets to learn small calibration maps between similar replicate distributions.
- Maximum Mean Discrepancy: MMD measures the distance between two probability distributions, using a function class such as a reproducing kernel Hilbert space.
- Maximum Mean Discrepancy: With a universal kernel, MMD equals zero exactly when the two distributions are equal, and its sample version operates on observed finite samples.
- Maximum Mean Discrepancy: MMD has been used as a neural-network loss for generative modeling, while this paper applies it to batch-effect calibration.
- Residual Nets: A ResNet block computes y = x + δ(x), combining an identity term with a learned residual transformation.
- Residual Nets: Shortcut connections support optimization and make functions close to the identity easier to learn, matching calibration between distributions that are already similar.
3 Tackling the Calibration Problem
The calibration problem is formulated as learning a continuous map from one distribution to another using paired finite samples. MMD-ResNet represents the map as a near-identity residual network and trains it to align source and target distributions.
- The method assumes a continuous map exists between source and target distributions and that this map is a small perturbation of identity.
- Given finite samples from both distributions, the goal is to learn a map whose transformed source sample is likely to come from the target distribution.
- Because the desired map is near identity, it is written as ψ(x) = x + δ(x), directly matching the residual-network form.
- MMD-ResNet receives source points and learns a map that makes their distribution similar to the target sample.
- Training uses a loss based on MMD computed stochastically on mini-batches from the source and target samples.
4 Experimental Results
Experiments on CyTOF data show that MMD-ResNet calibration brings source distributions closer to target distributions and preserves biological structure, with ResNet outperforming linear calibration methods.
- Experimental setup: CyTOF and single-cell RNA-seq are used to evaluate calibration across two high-throughput biological technologies.The paper also describes CyTOF as measuring multiple protein markers per cell and scRNA-seq as measuring gene expression in single cells.
- Experimental setup: CyTOF samples were collected in four source-target pairs from replicate specimens measured on different days.Day 1 samples were treated as source and day 2 samples as target; each sample had 25 dimensions and 1800–5000 cells.
- Pre-processing: After denoising and normalization, the source and target samples still differed in distribution before calibration.The preprocessing included manual filtering, log transformation, bead normalization, and a denoising autoencoder for zero-heavy measurements.
- CyTOF calibration: The calibrated data was significantly closer to the target in MMD, and ResNet matched MLP on two pairs while outperforming it on two others.Table 1 reports averages and standard deviations over five random subsets of 1000 cells.
- CyTOF calibration: Across individual markers, calibrated source empirical distributions were substantially closer to target distributions than uncalibrated source distributions.Figure 3 compares target, source, and calibrated source empirical cumulative distribution functions for the first six markers.
- Biological validation: ResNet preserved the CD8+T-cell region appropriately, whereas calibration without shortcut connections failed to map it to the target region.The comparison used the CD28 and GzB marker plane and contrasted pre-calibration, MLP, and ResNet outputs.
- Biological validation: Shortcut connections are important because minimizing MMD alone does not ensure an identity-like calibration map.Different low-MMD maps may be far from the identity and easier to reach from random initialization; residual networks are designed to learn maps near identity.
- Comparison to linear methods: MMD-ResNet outperformed marker-wise mean-variance matching and principal-component batch-effect removal in the reported comparison.The comparison is based on MMD scores in Table 2.
4.3 Calibration of Single-Cell RNA-seq Data
The authors apply MMD-ResNet to calibrate single-cell RNA-seq batches, where multiple runs are needed because each run captures too few usable cells. The calibrated source clusters align with target-batch counterparts and achieve lower MMD than competing methods.
- Drop-seq requires multiple batches because fewer than half of approximately 5000 cells per run typically contain enough genes for statistical analysis.
- MMD-ResNet maps source-batch clusters onto their target-batch counterparts in t-SNE visualizations after calibration.
- MMD-ResNet achieves lower MMD than mean-variance matching, PCA, and Combat in RNA calibration.
- Calibration appropriately preserves the subpopulation of cells with log-transformed Prkca expression values ≥3 and visually outperforms Combat.
4.4 Indirect Calibration
The authors test whether calibration can be learned indirectly by composing MMD-ResNets trained across different patient and instrument-day pairs. Indirect calibration removes much of the batch effect while remaining only slightly less accurate than direct calibration.
- Indirect calibration maps p1d1 to p1d2 through networks trained on cross-patient and cross-day replicate mappings.
- MMD decreases from 0.69 before calibration to 0.27 after direct calibration and 0.29 after indirect calibration.
- Indirect calibration is only slightly less accurate than direct calibration and removes much of the batch effect.
- The results suggest that the approach can calibrate samples when reference replicates span two batches and additional samples occur in only one batch.
- A generalized setup can map samples from multiple days to coordinates of a single day-0 reference sample.
5 Related Work
The related-work discussion contrasts MMD-ResNet with existing biological calibration methods and positions MMD-based neural networks as a new approach to batch-effect removal.
- Existing approaches include bead normalization and warping for cytometry, as well as surrogate-variable analysis for gene-expression data.
- Bead normalization may leave substantial distribution differences between CyTOF samples, while surrogate-variable analysis is designed for supervised settings with phenotype labels.
- MMD has been used as a neural-network loss for generative modeling, but the authors apply MMD nets to batch-effect removal.
6 Discussion
The discussion notes that MMD may better evaluate calibration by accounting for internal sample structure, while cluster-composition differences constrain continuous-map calibration. Subsampling is suggested when corresponding cluster proportions differ.
- MMD may be more suitable than Parzen estimates because it accounts for the internal structure of the calibrated source sample.
- MMD-ResNet is not expected to account for differences in relative proportions of corresponding cell-type clusters because it computes a continuous map.
- When cluster proportions differ between source and target samples, the authors suggest subsampling to match each cell type's relative proportion.
7 Conclusion
The paper presents a residual-network approach for nonlinear batch-effect removal that matches source and target distributions across biological data types. It also introduces indirect calibration and reports a limitation: calibrated samples can still fail a formal two-sample distribution test.
- Residual networks remove batch effects nonlinearly by matching the distributions of source and target samples.The approach was applied to CyTOF and single-cell RNA-seq data.
- The approach achieved impressive performance on CyTOF and single-cell RNA-seq data, with CyTOF performance described as previously unreported.
- Indirect calibration uses maps between replicates from other specimens and can later apply the learned map to new data.
- Shortcut connections preserve biological conditions, whereas equivalent networks without them may distort those conditions.
- A two-sample test using MMD can still reject equality between the calibrated source and target distributions.
A Specification of Markers in CyTOF Experiments
The appendix identifies the 25 markers used to characterize cell sub-populations in the CyTOF experiments.
- Table 4 provides information about the 25 markers used in the CyTOF experiments.
B Additional Plots for CyTOF calibration
The appendix adds CyTOF calibration plots for additional source-target pairs and specifies the markers used in the experiments. The figures compare samples before and after calibration, including a residual-network comparison with a network lacking shortcut connections.
- Additional CyTOF calibration plots: Figure 9 projects source and target CyTOF samples onto the first two principal components of the target data for three additional source-target pairs.The plots compare samples before calibration with samples after calibration.
- Additional CyTOF calibration plots: Figure 10 shows CD8+ T-cell sub-populations in the (CD28,GzB) plane for the three additional source-target pairs.Each row compares before calibration, ResNet calibration, and calibration with an identical network without shortcut connections.
- Marker specification: The CyTOF experiments use 25 markers to characterize cell sub-populations, with embedded bead standards enabling bead normalization.