Source-linked AI summary
Coarse to Fine: Iterative Adversarial Neural Cellular Automata for Medical Image Synthesis
Anh Thi Luu, Nick Lemke, Anirban Mukhopadhyay
TL;DR
Medical image sharing is constrained by privacy and legal restrictions, while high-quality generative models can be computationally expensive. StyleGANCA combines StyleGAN-inspired latent control with multi-scale NCA synthesis to generate medical images efficiently, achieving strong benchmark performance and preserving class-specific information for downstream classification.
Problem
Privacy and legal restrictions limit medical image sharing, while existing high-quality generative models can be computationally expensive for resource-constrained hardware.
Method
StyleGANCA integrates a StyleGAN-inspired mapping network and adaptive style modulation into a multi-scale NCA synthesis process for latent-controlled iterative image generation.
Results
StyleGANCA outperforms baseline approaches on PathMNIST and remains competitive on BloodMNIST with 617k trainable parameters, while generated images support downstream classification.
Takeaways & Limitations
The generated images preserve meaningful semantic and class-discriminative information, supporting StyleGANCA as an efficient alternative for controllable medical image synthesis.
Takeaways & Limitations
For unconditional generative baselines, generated samples are pseudo-labeled using classifiers trained on the corresponding real training sets.
Abstract
from arXiv · showhide
Large-scale, publicly available datasets have driven advances in deep learning, but privacy and legal restrictions often limit data sharing in medical imaging. Synthetic data generation offers a privacy-friendly alternative to enable the training of high-performance models on health data. While most state-of-the-art generative models produce high-quality images, they remain computationally expensive, which limits their applicability on resource-constrained hardware. We propose StyleGANCA, the first lightweight general-purpose NCA-based generative adversarial network. The architecture integrates a StyleGAN-inspired mapping network and adaptive style modulation into a multi-scale NCA synthesis process, enabling latent-controlled image generation through iterative local interactions. We evaluate StyleGANCA on BloodMNIST and PathMNIST against adversarial, variational, diffusion, and NCA-based baselines. Experimental results demonstrate that StyleGANCA achieves competitive image quality with substantially fewer parameters than baseline architectures, achieving the best FID and KID scores on PathMNIST with only 617k parameters. Furthermore, downstream experiments show that the generated images preserve class-specific information and effectively support the training of multi-class classifiers. Our code is publicly available at: https://github.com/MECLabTUDA/StyleGANCA
1 Introduction
Medical imaging data are valuable for robust model training but difficult to share because of legal, ethical, and privacy constraints. StyleGANCA addresses the need for efficient synthetic generation by combining style-based latent control with multi-scale neural cellular automata.
- Legal, ethical, and privacy regulations make collecting and sharing large medical imaging datasets across institutions challenging.
- GANs provide high-quality images at moderate computational cost, but increasingly large architectures motivate more parameter-efficient generative models.
- NCAs construct images through repeated local interactions, offering lightweight and scalable models, but existing approaches show quality, generalization, or speed limitations.
- StyleGANCA combines StyleGAN-inspired latent control with hierarchical multi-scale NCAs for iterative image synthesis.A mapping network transforms latent noise into an intermediate representation that controls features at each synthesis block through adaptive group normalization.
- The study evaluates StyleGANCA and a class-conditional variant on PathMNIST and BloodMNIST against diverse generative baselines.
2 Methodology
StyleGANCA replaces a conventional feed-forward synthesis network with hierarchical NCAs operating across resolutions. Its mapping network, noise injection, style modulation, and repeated local updates provide latent-controlled progressive generation.
- StyleGANCA combines a latent mapping network with a multi-scale NCA synthesis network trained adversarially.
- Latent Mapping Network: The mapping network transforms z into an intermediate representation w whose style information controls synthesis at every resolution level.
- Multi-Scale NCA-based Synthesis Network: Synthesis begins from a learned 4×4 constant state, then progressively refines and upsamples representations from global structure to fine-grained detail.
- Multi-Scale NCA-based Synthesis Network: Noise injection adds Gaussian perturbations with learnable per-channel scaling to introduce stochastic variation in fine-grained details.
- Multi-Scale NCA-based Synthesis Network: Adaptive style modulation converts w into scale and bias parameters, normalizes the hidden state, and applies style-dependent modulation.
- Multi-Scale NCA-based Synthesis Network: Repeated NCA updates use neighborhood information and shared neural functions, with lower resolutions capturing coarse structure and higher resolutions refining textures.
- Conditional Variant: The conditional variant injects a label embedding before the mapping network and supplies corresponding label information to the discriminator.
3 Experimental Setup
The experiments evaluate StyleGANCA on two 64×64 medical image datasets using image-quality, parameter-efficiency, and downstream classification measures. Comparisons include variational, diffusion, GAN, and NCA-based generators.
- Datasets: BloodMNIST contains eight blood-cell classes, while PathMNIST contains nine colorectal histopathology tissue classes.
- Datasets: Generator training uses 64×64 images, whereas downstream classifiers use 28×28 images under the MedMNIST evaluation protocol.
- Training Details: StyleGANCA is trained with WGAN-GP, Adam optimization, batch size 32, exponential moving-average evaluation, and five hierarchical synthesis blocks.
- Evaluation Metrics: Image quality is measured with FID and KID, while downstream classification is evaluated using AUC, accuracy, and F1 score.
- Evaluation Metrics: Figure 3 plots FID and KID against trainable parameter count for PathMNIST and BloodMNIST.
- Evaluation Metrics: Unconditional generative baselines use pseudo-labels produced by classifiers trained on the corresponding real training sets.
- Baselines: The comparison includes VAE, DDPM, DiT, DCGAN, StyleGAN, VNCA, and FourierDiff-NCA baselines.
4 Results
StyleGANCA delivers competitive or state-of-the-art medical image synthesis with a compact architecture, while its generated images retain class-discriminative information for downstream classification. Its multi-scale synthesis progressively builds coarse structures and fine textures, and hyperparameter increases improve quality at added computational cost.
- Quantitative Comparison: StyleGANCA achieves the best overall performance on PathMNIST and competitive performance on BloodMNIST with only 617k parameters.The conditional variant achieves the lowest FID on PathMNIST while maintaining a similarly low KID score.
- Qualitative Comparison: Generated StyleGANCA images are visually comparable to those from significantly larger models while retaining NCA-like parameter efficiency.The multi-scale process establishes coarse structures before iteratively refining fine-grained textures.
- Classification Suitability: StyleGANCA-trained classifiers achieve AUCs of 0.955 and 0.988 and accuracies of 0.761 and 0.880 on PathMNIST and BloodMNIST, respectively.The generated images preserve meaningful class-discriminative information for downstream multi-class learning.
- Ablation Study: Increasing latent dimension, NCA state channels, or update steps consistently improves FID and KID while increasing model complexity or GPU memory use.The ablation supports adapting StyleGANCA to different resource constraints through a quality-efficiency trade-off.
5 Conclusion
StyleGANCA combines style-based latent control with multi-scale neural cellular automata to enable controllable, lightweight medical image synthesis. Across BloodMNIST and PathMNIST, it balances image quality and model complexity while preserving coherent visual and class-discriminative information.
- Contribution: StyleGANCA combines a StyleGAN-inspired mapping network and adaptive style modulation with multi-scale NCAs for controllable image generation through iterative local interactions.The framework uses 617k parameters.
- Results: On BloodMNIST and PathMNIST, StyleGANCA achieves a good image-quality and model-complexity trade-off with 617k trainable parameters.The model outperforms baseline approaches on PathMNIST and remains competitive on BloodMNIST.
- Implications: Qualitative and downstream experiments show realistic images with coherent structures and fine-grained textures that preserve meaningful semantic and class-discriminative information.These findings support StyleGANCA as a potential alternative for controllable and efficient image synthesis.