PaperlayerView exact revision on arXiv

Source-linked AI summary

GANs for Biological Image Synthesis

Anton Osokin, Anatole Chessel, Rafael E. Carazo Salas, Federico Vaggi

arXiv:1708.04692v2cs.CVcs.LGstat.ML

TL;DR

Fluorescence screens limit how many proteins can be imaged together, while the paper seeks to model co-localization of independently imaged proteins and their localization changes through the cell cycle. It adapts GANs to generate multi-channel images with channel dependence, and latent-space interpolation reproduces a reported Arp3 localization shift during mitosis.

  • Problem

    The paper targets the limited ability of fluorescent screens to image multiple proteins together and the challenge of studying protein-localization changes across the cell cycle.

    Source: §1 · p. 2

  • Method

    The authors modify GANs to make green-channel generation causally dependent on red and propose nearest-neighbor multi-channel training and a star-shaped generator.

    Source: §1 · p. 2

  • Results

    Latent-space interpolation moves through cell-cycle stages, with Arp3 gradually shifting its dot-like pattern from cell tips toward the middle during mitosis, as previously described.

    Source: §5.3 · p. 8

  • Takeaways & Limitations

    The authors suggest channel-correlation-based simulations could support future exploration of uninvestigated, inaccessible, or unaffordable biological and biomedical experiments.

    Source: §6 · p. 8

    The authors cannot compare generated multi-channel images with real fluorescent images because such multi-channel images are unavailable.

    Source: §5.3 · p. 8

Abstract

from arXiv · show

In this paper, we propose a novel application of Generative Adversarial Networks (GAN) to the synthesis of cells imaged by fluorescence microscopy. Compared to natural images, cells tend to have a simpler and more geometric global structure that facilitates image generation. However, the correlation between the spatial pattern of different fluorescent proteins reflects important biological functions, and synthesized images have to capture these relationships to be relevant for biological applications. We adapt GANs to the task at hand and propose new models with casual dependencies between image channels that can generate multi-channel images, which would be impossible to obtain experimentally. We evaluate our approach using two independent techniques and compare it against sensible baselines. Finally, we demonstrate that by interpolating across the latent space we can mimic the known changes in protein localization that occur through time during the cell cycle, allowing us to predict temporal evolution from static images.

1. Introduction

The paper targets two limits of fluorescent imaging screens: only a limited number of proteins can be imaged together, and protein localization changes through the cell cycle. It adapts GANs to synthesize multi-channel cell images and study these localization patterns.

1. Introduction

  • Fluorescence microscopy enables high-resolution observation of biological events, while the resulting data deluge motivates quantitative bioimage analysis.The passage identifies bioimage informatics as a source of novel data-analysis problems that machine learning could help solve.Source: §1 · p. 1
  • The star-shaped GAN generates multiple green protein channels aligned with one Bgs4-red image, although training images contain only one green channel.Figure 1 shows real and generated fission-yeast images.Source: Figure 1 · p. 1, §1 · p. 2
  • Prior GAN applications cited here focus on natural-image datasets, and the authors report no known GAN applications to biological images.The paper situates its work amid recent progress in generative models for natural images.Source: §1 · p. 1
  • The paper aims to use red-channel information to synthesize cells with multiple independently imaged green-labeled proteins and study their localization changes through the cell cycle.The goal is to predict how protein localizations might co-exist if imaged together, despite limits on how many signal channels can be imaged at once.Source: §1 · p. 2
  • The authors modify DCGAN with causal dependence of green on red, use WGAN-GP to address mode collapse, and propose nearest-neighbor and star-shaped multi-channel approaches.They evaluate the models with a neural-network two-sample test and held-out test-set reconstruction.Source: §1 · p. 2

2. Related Work

The paper draws on work addressing GAN training and conditional generation, then distinguishes its channel-separation design from unsupervised discovery of latent factors. It places this approach among prior biological image-generation studies.

2. Related Work

  • The authors combine DCGAN with the Wasserstein loss to help address mode collapse, which appeared especially in their separable setting.GAN training can also suffer oscillations, and prior work addressed these issues through architecture and training-objective changes.Source: §2 · p. 2
  • Pix2pix produced high-quality green images given red inputs on LIN, but could not generate multiple realistic green images for one red input.The authors connect this conditional mode collapse to datasets with only one target image per source, which provides indirect supervision for variety.Source: §2 · p. 2
  • Unlike unsupervised methods that encourage latent-factor disentanglement [4], this work explicitly embeds factor separation in the model.The paper's generator gives red and green channels independent noise, while allowing red—but not green—to influence green generation.Source: §2 · p. 2
  • Prior biological generative models include work generating cell puncta patterns conditional on microtubule localization, while the authors describe their study as the first reported GAN use on fluorescent-microscopy samples.The cited prior work is presented as demonstrating the potential of generative models for studying relative sub-cellular protein positions.Source: §2 · p. 3

3. Biological Background

The biological background explains what fluorescent channels measure, how fission-yeast growth relates to cell-cycle stage, and the composition and scope of the LIN dataset.

3.1. Fluorescent Imaging

  • Fluorescent proteins mark proteins of interest for imaging, but only 3–4 proteins can be tagged and imaged simultaneously because suitable non-overlapping spectra are limited.In fluorescent images, channel intensity reflects local tagged-protein concentration, and cross-channel correlations carry information about relationships between proteins.Source: §3.1 · p. 3

3.2. Fission Yeast Cells

  • Fission yeast grow from about 7 µm to 14 µm in length, and cell length is a good proxy for age through most of the cell cycle.Bgs4 marks areas of active growth, so its localization helps infer growth-cycle stage and where growth occurs.Source: §3.2 · p. 3

3.3. The LIN Dataset

  • The LIN dataset contains around 170,000 centered fission-yeast cell images, with Bgs4 in red and one of 41 polarity-regulating proteins in green.The paper's experiments use six polarity factors, yielding 26,909 images center-cropped and resized to 48 × 80 pixels.Source: §3.3 · p. 3

4. GANs for Image Generation

The section reviews GAN and WGAN-GP training, then develops channel-separated and multi-channel generators for fluorescence images. Its extensions use red-channel information to coordinate green outputs and address the limitations of constructing multi-channel training data.

4.1. Preliminaries

  • GAN training is a two-player game: the generator creates images from noise while the discriminator distinguishes generated images from training images.The discriminator maximizes the objective and the generator minimizes it, with the optimizers alternating updates.Source: §4.1 · pp. 3–4
  • The section builds on DCGAN’s up-convolutional generator and describes Wasserstein distance as the cost of an optimal transport between data and generated distributions.The Wasserstein formulation addresses cases where Jensen–Shannon divergence provides no useful training direction.Source: §4.1 · p. 4
  • WGAN-GP approximates the 1-Lipschitz constraint with a gradient-norm penalty and is used for its better convergence properties than the original GAN objective [15].The earlier WGAN approach clips discriminator parameters; WGAN-GP instead penalizes deviations of the discriminator gradient norm from one.Source: §4.1 · p. 4

4.2. Model Extensions

  • Channel separation assigns half the upconvolutional filters to red and half to green, with one-way connections from red filters to green filters.This design is intended to make the green output match the red channel.Source: §4.2 · p. 5
  • The artificial multi-channel training set pairs each image with nearest-red green channels from other classes, but those channels can remain inconsistent with the original red.The proposed multi-output DCGAN generates c+1 channels from this constructed set.Source: §4.2 · p. 5
  • The star-shaped generator uses one red tower to feed c green towers, with green outputs independent conditional on the red channel.The model can be trained on existing two-channel images; it uses c discriminators paired with the same generated red, whose gradients accumulate for the generator.Source: §4.2 · p. 5
  • Applying batch normalization c times in the shared red tower causes interference between normalizations and prevents convergence.The authors report that using batch normalization only once in that tower was important.Source: §4.2 · p. 5

5. Experiments

The experiments assess image quality and mode coverage with two evaluation approaches, then use latent-space interpolation to examine cell-cycle localization changes. The results favor individual one-class models for quality, with star-shaped models best among synchronized-channel models and limited degradation in reconstruction except for hard proteins.

5.1. Neural-network Two-sample Test

  • C2ST scores track image quality across training, and WGAN-GP samples look better than GAN samples by visual inspection and cross-entropy C2ST.Figure 4 reports scores of 3.2 ± 0.1 for GAN and 1.6 ± 0.1 for WGAN-GP, alongside greater green-channel variability conditioned on red for WGAN-GP.Source: §5.1 · p. 6, Figure 4 · p. 6, Figure 3 · p. 6
  • C2ST gives lower scores for known-similar protein pairs than for most other pairs, while scores remain higher than comparisons of a protein with itself.The experiment also compares real images from different classes; C2ST is not a metric, and Table 1 is asymmetric because of train/test split biases.Source: §5.1 · p. 6
  • Individual one-class WGAN-GP models produce higher-quality images than synchronized-channel models; among synchronized models, star-shaped models perform best, with significant quality drops for some proteins.All compared models use WGAN-GP with the same generator-width-to-output-channel ratio.Source: §5.1 · p. 7

5.2. Optimization to Reconstruct the Test Set

  • The reconstruction evaluation minimizes pixel-normalized L2 error over latent noise, with regular and separable procedures testing image reconstruction and green-channel modes conditioned on red.The study also reports latent-vector NLL, using nearest-neighbor error and Gaussian-prior noise NLL as baselines.Source: §5.2 · p. 7
  • Separable reconstruction is harder than regular reconstruction, while WGAN-GP reconstructs better, probably because it suffers less from mode collapse.Star-shaped models do not degrade reconstruction performance except for some hard proteins.Source: §5.2 · pp. 7–8

5.3. Progression Through the Cell Cycle

  • Latent-space interpolation with the separable GAN architecture traces growth and division while keeping green-channel output consistent with red; Arp3 shifts its tip-localized dot pattern toward the cell middle during mitosis.The Arp3 movement matches a previously described pattern.Source: §5.3 · p. 8
  • The LIN dataset lacks true images with three or more channels, so the generated multi-channel images cannot be compared against real fluorescent images.The authors identify more open biological datasets as a needed baseline for comparison.Source: §5.3 · p. 8

6. Conclusion

The paper argues that generative models can be applied fruitfully to fluorescent-microscopy images. By leveraging correlations between channels, the approach simulates multiple proteins’ localization through the cell cycle, with possible future uses in biological and biomedical research.

6. Conclusion

  • The results suggest modern generative models can be applied fruitfully to images obtained by fluorescent microscopy.Source: §6 · p. 8
  • Leveraging correlations between image channels allowed simulation of multiple proteins’ localization throughout the cell cycle.Source: §6 · p. 8
  • The approach could enable future exploration of inaccessible biological experiments and bioimaging applications.The passage also names uninvestigated or unaffordable experiments, new discoveries, and diagnostic or prognostic applications as possibilities.Source: §6 · p. 8

A. Comparison of C2ST variants

The comparison finds that WGAN and WGAN-GP generators outperform GAN generators under all tested C2ST variants. WGAN-based C2ST scores are more stable, with no clear winner between WGAN and WGAN-GP.

A. Comparison of C2ST variants

  • The comparison applies GAN-, WGAN-, and WGAN-GP-based discriminator training to regular and separable generators at different training stages.For GAN, the score is negative cross-entropy; WGAN uses clipped weights and the negated objective; WGAN-GP uses a gradient-norm regularizer with weight 10.Source: A. Comparison of C2ST variants · p. 11
  • C2ST optimizers run for 5,000 iterations, using Adam for GAN and WGAN-GP and RMSprop for WGAN.Source: A. Comparison of C2ST variants · p. 11
  • All C2ST variants indicate WGAN and WGAN-GP generators are significantly better than GAN generators, while WGAN-based scores are more stable and show no clear winner.Source: A. Comparison of C2ST variants · p. 11

B. Reconstructing images of different proteins

Across proteins, WGAN-GP models reconstruct consistently better than GAN-objective models on both NLL and L2 error. Mkh1 is the difficult exception for star-shaped models, while other proteins can be reconstructed as well as with simpler separable models.

B. Reconstructing images of different proteins

  • The extended reconstruction results include NLL-versus-error scatter plots for six selected proteins and example reconstructions for each.Examples include test images, L2 nearest neighbors, regular reconstructions, and separable reconstructions.Source: B. Reconstructing images of different… · p. 11, Figure 11 · p. 14
  • WGAN-GP models consistently achieve smaller NLL and L2 reconstruction errors than models trained with the GAN objective.Source: Figure 10 · p. 13
  • For Mkh1, the star-shaped separable GAN model reconstructs very poorly, while its WGAN-GP counterpart has high NLL; other proteins reconstruct as well as with simpler separable models.Source: Figure 10 · p. 13
Loading 1708.04692v2…