PaperlayerView exact revision on arXiv

Source-linked AI summary

Removal of Batch Effects using Distribution-Matching Residual Networks

Uri Shaham, Kelly P. Stanton, Jun Zhao, Huamin Li, Khadir Raddassi, Ruth Montgomery, Yuval Kluger

arXiv:1610.04181v6stat.ML

TL;DR

Systematic instrument-related batch effects can distort analyses of technologies such as CyTOF and single-cell RNA-seq. The paper uses a residual network trained with Maximum Mean Discrepancy to align replicate distributions, and reports effective attenuation of batch effects across both data types.

  • Problem

    Systematic measurement errors in technologies such as CyTOF and single-cell RNA-seq can affect statistical analysis when batch effects are not calibrated.

    Source: §1 · p. 1, §4.3 · p. 11

  • Method

    A residual network is trained to minimize Maximum Mean Discrepancy between source and target replicate distributions measured in different batches.

    Source: §1 · p. 2, §7 · p. 16

  • Results

    The approach successfully removes significant portions of batch effects in CyTOF and single-cell RNA-seq and outperforms other popular calibration approaches.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Residual networks preserve biological properties during calibration, while networks without shortcut connections might distort them.

    Source: §1 · p. 2, §4.2.4 · p. 10

    A two-sample test still rejects equality between the calibrated source and target distributions.

    Source: §7 · p. 16

Abstract

from arXiv · show

Sources of variability in experimentally derived data include measurement error in addition to the physical phenomena of interest. This measurement error is a combination of systematic components, originating from the measuring instrument, and random measurement errors. Several novel biological technologies, such as mass cytometry and single-cell RNA-seq, are plagued with systematic errors that may severely affect statistical analysis if the data is not properly calibrated. We propose a novel deep learning approach for removing systematic batch effects. Our method is based on a residual network, trained to minimize the Maximum Mean Discrepancy (MMD) between the multivariate distributions of two replicates, measured in different batches. We apply our method to mass cytometry and single-cell RNA-seq datasets, and demonstrate that it effectively attenuates batch effects.

1 Introduction

The paper addresses systematic batch effects that make replicate biological measurements differ for irrelevant instrument-related reasons. It proposes MMD-ResNet calibration and evaluates it on CyTOF and single-cell RNA-seq data.

  • Batch effects are distributional differences between replicate measurements caused by irrelevant instrument-related factors.They introduce systematic error that can produce spurious analyses or obscure the signal of interest.Source: §1 · pp. 1–2
  • CyTOF replicate specimens measured in different batches can differ noticeably in multivariate protein space, requiring calibration for valid analysis.Source: §1 · p. 2
  • MMD-ResNet learns a map that calibrates a source distribution toward a target distribution when their difference is moderate and near the identity.The source and target are designated from replicate samples measured in different batches.Source: §1 · p. 2
  • The approach removes significant portions of batch effects in CyTOF and single-cell RNA-seq and outperforms other popular calibration approaches.The authors report that comparable CyTOF performance had not previously been reported to their knowledge.Source: §1 · p. 2
  • Shortcut connections are crucial because networks without them might distort biological properties, while indirect calibration can connect replicates through other specimens.Source: §1 · p. 2

2 Preliminaries

The preliminaries define MMD as a distributional distance and describe residual networks as identity-preserving architectures. These properties motivate using ResNets to learn small calibration maps between similar replicate distributions.

Maximum Mean Discrepancy

  • MMD measures the distance between two probability distributions, using a function class such as a reproducing kernel Hilbert space.Source: §2.1 · p. 3
  • With a universal kernel, MMD equals zero exactly when the two distributions are equal, and its sample version operates on observed finite samples.Source: §2.1 · p. 3
  • MMD has been used as a neural-network loss for generative modeling, while this paper applies it to batch-effect calibration.Source: §2.1 · p. 4

Residual Nets

  • A ResNet block computes y = x + δ(x), combining an identity term with a learned residual transformation.Source: §2.2 · p. 4
  • Shortcut connections support optimization and make functions close to the identity easier to learn, matching calibration between distributions that are already similar.Source: §2.2 · p. 5

3 Tackling the Calibration Problem

The calibration problem is formulated as learning a continuous map from one distribution to another using paired finite samples. MMD-ResNet represents the map as a near-identity residual network and trains it to align source and target distributions.

  • The method assumes a continuous map exists between source and target distributions and that this map is a small perturbation of identity.Source: §3 · p. 5
  • Given finite samples from both distributions, the goal is to learn a map whose transformed source sample is likely to come from the target distribution.Source: §3 · p. 5
  • Because the desired map is near identity, it is written as ψ(x) = x + δ(x), directly matching the residual-network form.Source: §3 · p. 5
  • MMD-ResNet receives source points and learns a map that makes their distribution similar to the target sample.Source: §3 · p. 5
  • Training uses a loss based on MMD computed stochastically on mini-batches from the source and target samples.Source: §3 · p. 5

4 Experimental Results

Experiments on CyTOF data show that MMD-ResNet calibration brings source distributions closer to target distributions and preserves biological structure, with ResNet outperforming linear calibration methods.

Experimental setup

  • CyTOF and single-cell RNA-seq are used to evaluate calibration across two high-throughput biological technologies.The paper also describes CyTOF as measuring multiple protein markers per cell and scRNA-seq as measuring gene expression in single cells.Source: §4 · pp. 5–6
  • CyTOF samples were collected in four source-target pairs from replicate specimens measured on different days.Day 1 samples were treated as source and day 2 samples as target; each sample had 25 dimensions and 1800–5000 cells.Source: §4.2.1 · pp. 6–7

Pre-processing

  • After denoising and normalization, the source and target samples still differed in distribution before calibration.The preprocessing included manual filtering, log transformation, bead normalization, and a denoising autoencoder for zero-heavy measurements.Source: §4.2.2 · p. 7

CyTOF calibration

  • The calibrated data was significantly closer to the target in MMD, and ResNet matched MLP on two pairs while outperforming it on two others.Table 1 reports averages and standard deviations over five random subsets of 1000 cells.Source: Table 1 · p. 8, §4.2.3 · p. 8
  • Across individual markers, calibrated source empirical distributions were substantially closer to target distributions than uncalibrated source distributions.Figure 3 compares target, source, and calibrated source empirical cumulative distribution functions for the first six markers.Source: §4.2.3 · p. 8, Figure 3 · p. 9

Biological validation

  • ResNet preserved the CD8+T-cell region appropriately, whereas calibration without shortcut connections failed to map it to the target region.The comparison used the CD28 and GzB marker plane and contrasted pre-calibration, MLP, and ResNet outputs.Source: §4.2.4 · p. 8, Figure 4 · p. 10
  • Shortcut connections are important because minimizing MMD alone does not ensure an identity-like calibration map.Different low-MMD maps may be far from the identity and easier to reach from random initialization; residual networks are designed to learn maps near identity.Source: §4.2.4 · pp. 8, 10

Comparison to linear methods

  • MMD-ResNet outperformed marker-wise mean-variance matching and principal-component batch-effect removal in the reported comparison.The comparison is based on MMD scores in Table 2.Source: §4.2.5 · p. 10

4.3 Calibration of Single-Cell RNA-seq Data

The authors apply MMD-ResNet to calibrate single-cell RNA-seq batches, where multiple runs are needed because each run captures too few usable cells. The calibrated source clusters align with target-batch counterparts and achieve lower MMD than competing methods.

  • Drop-seq requires multiple batches because fewer than half of approximately 5000 cells per run typically contain enough genes for statistical analysis.Source: §4.3 · pp. 10–11
  • MMD-ResNet maps source-batch clusters onto their target-batch counterparts in t-SNE visualizations after calibration.Source: §4.3 · p. 11, Figure 5 · p. 12
  • MMD-ResNet achieves lower MMD than mean-variance matching, PCA, and Combat in RNA calibration.Source: §4.3 · p. 11, Table 3 · p. 12
  • Calibration appropriately preserves the subpopulation of cells with log-transformed Prkca expression values ≥3 and visually outperforms Combat.Source: §4.3 · p. 12

4.4 Indirect Calibration

The authors test whether calibration can be learned indirectly by composing MMD-ResNets trained across different patient and instrument-day pairs. Indirect calibration removes much of the batch effect while remaining only slightly less accurate than direct calibration.

  • Indirect calibration maps p1d1 to p1d2 through networks trained on cross-patient and cross-day replicate mappings.Source: §4.4 · pp. 12–13, Figure 7 · p. 13
  • MMD decreases from 0.69 before calibration to 0.27 after direct calibration and 0.29 after indirect calibration.Source: §4.4 · p. 13
  • Indirect calibration is only slightly less accurate than direct calibration and removes much of the batch effect.Source: §4.4 · p. 13, Figure 8 · p. 14
  • The results suggest that the approach can calibrate samples when reference replicates span two batches and additional samples occur in only one batch.Source: §4.4 · pp. 13, 15
  • A generalized setup can map samples from multiple days to coordinates of a single day-0 reference sample.Source: §4.4 · p. 15

5 Related Work

The related-work discussion contrasts MMD-ResNet with existing biological calibration methods and positions MMD-based neural networks as a new approach to batch-effect removal.

  • Existing approaches include bead normalization and warping for cytometry, as well as surrogate-variable analysis for gene-expression data.Source: §5 · p. 15
  • Bead normalization may leave substantial distribution differences between CyTOF samples, while surrogate-variable analysis is designed for supervised settings with phenotype labels.Source: §5 · p. 15
  • MMD has been used as a neural-network loss for generative modeling, but the authors apply MMD nets to batch-effect removal.Source: §5 · p. 15

6 Discussion

The discussion notes that MMD may better evaluate calibration by accounting for internal sample structure, while cluster-composition differences constrain continuous-map calibration. Subsampling is suggested when corresponding cluster proportions differ.

  • MMD may be more suitable than Parzen estimates because it accounts for the internal structure of the calibrated source sample.Source: §6 · pp. 15–16
  • MMD-ResNet is not expected to account for differences in relative proportions of corresponding cell-type clusters because it computes a continuous map.Source: §6 · p. 16
  • When cluster proportions differ between source and target samples, the authors suggest subsampling to match each cell type's relative proportion.Source: §6 · p. 16

7 Conclusion

The paper presents a residual-network approach for nonlinear batch-effect removal that matches source and target distributions across biological data types. It also introduces indirect calibration and reports a limitation: calibrated samples can still fail a formal two-sample distribution test.

  • Residual networks remove batch effects nonlinearly by matching the distributions of source and target samples.The approach was applied to CyTOF and single-cell RNA-seq data.Source: §7 · p. 16
  • The approach achieved impressive performance on CyTOF and single-cell RNA-seq data, with CyTOF performance described as previously unreported.Source: §7 · p. 16
  • Indirect calibration uses maps between replicates from other specimens and can later apply the learned map to new data.Source: §7 · p. 16
  • Shortcut connections preserve biological conditions, whereas equivalent networks without them may distort those conditions.Source: §7 · p. 16
  • A two-sample test using MMD can still reject equality between the calibrated source and target distributions.Source: §7 · p. 16

A Specification of Markers in CyTOF Experiments

The appendix identifies the 25 markers used to characterize cell sub-populations in the CyTOF experiments.

B Additional Plots for CyTOF calibration

The appendix adds CyTOF calibration plots for additional source-target pairs and specifies the markers used in the experiments. The figures compare samples before and after calibration, including a residual-network comparison with a network lacking shortcut connections.

Additional CyTOF calibration plots

Marker specification

  • The CyTOF experiments use 25 markers to characterize cell sub-populations, with embedded bead standards enabling bead normalization.Source: Table 4 · p. 22

Related papers

Loading 1610.04181v6…