Source-linked AI summary
Robust Registration of Calcium Images by Learned Contrast Synthesis
John A. Bogovic, Philipp Hanslovsky, Allan Wong, Stephan Saalfeld
TL;DR
Multi-modal registration of calcium-imaging brain volumes is difficult because subject and template images differ in modality and image properties. The paper learns a subject-to-template contrast mapping from manually registered pairs and uses the synthesized image for registration, improving robustness and reducing unusable outcomes.
Problem
Multi-modal registration remains challenging because images measuring similar quantities can differ in image content, noise properties, and dynamic range.
Method
The method learns a classifier-based mapping from subject intensities to template intensities and registers the resulting synthetic image with standard algorithms.
Results
Learning to synthesize the template modality produced more robust registration performance, with fewer subject registrations yielding unusable results.
Takeaways & Limitations
Cross correlation outperformed the other tested cost functions for these image data.
Takeaways & Limitations
The method was trained only on subjects for which baseline registration succeeded, and larger, more diverse training sets may further improve robustness.
Abstract
from arXiv · showhide
Multi-modal image registration is a challenging task that is vital to fuse complementary signals for subsequent analyses. Despite much research into cost functions addressing this challenge, there exist cases in which these are ineffective. In this work, we show that (1) this is true for the registration of in-vivo Drosophila brain volumes visualizing genetically encoded calcium indicators to an nc82 atlas and (2) that machine learning based contrast synthesis can yield improvements. More specifically, the number of subjects for which the registration outright failed was greatly reduced (from 40% to 15%) by using a synthesized image.
1. INTRODUCTION
The paper addresses registration of calcium-imaging brain volumes to an nc82 atlas, where modality differences and image quality make alignment challenging. Registration supports reliable neuron identification across large, heterogeneous neuronal populations.
- Registration helps identify activated neurons within large, heterogeneous populations by aligning experimental brains to a reference.
- Template and experimental images measure similar quantities but differ in modality, image content, noise properties, and dynamic range.
- Multi-modal image registration is challenging in medical and biological imaging because complementary images can have substantially different appearances.
- 1.1. Related work: Class-label mutual-information registration suits corresponding pixel classes but may struggle when label boundaries are unclear or image content differs substantially.
- 1.1. Related work: Prior synthesis work estimated a target modality from source patches using registered same-subject atlas pairs and reported improved intra-modality registration.
2. METHODS
The method learns a subject-to-template intensity mapping from manually registered image pairs, then registers the synthesized template-like image with SyN. It combines biological image acquisition, preprocessing, classifier-based synthesis, and deformable registration.
- 2. METHODS: The acquired calcium volumes use GCaMP6s in Fruitless neurons, while the bridging template uses a membrane label and nc82 neuropil staining.
- 2. METHODS: Baseline registration applies SyN to preprocessed subject and template images using SSD, normalized cross correlation, or mutual information.Preprocessing clips intensities above the 99th percentile, smooths with a 1.5 × 1.5 × 1.5 µm Gaussian kernel, and scales intensities to [0, 255].
- 2. METHODS: Manually registered subject–template pairs provide the training data for learning a mapping from subject intensities to template intensities.The mapping is learned in subject space to avoid upsampling the lower-resolution subject image.
- 2.4. Learning the synthesis mapping: Boosted decision trees and random forests learn the source-to-target modality mapping from image features.Experiments use local 5 × 5 × 3 intensity patches or multi-scale gradient, intensity, and texture features.
- 2.4. Learning the synthesis mapping: 200k samples from two training subjects are used, and the classifier predicts one of 10 binned template-intensity classes rather than continuous intensity.
- 2.4. Learning the synthesis mapping: The learned classifier is applied to every subject-image pixel to create a synthetic template image, which is registered using SyN with the baseline parameters.
3. EXPERIMENTS
Synthesis improved the qualitative usefulness and robustness of registration relative to the baseline, with fewer catastrophic landmark-transfer errors. The experiments compare training and held-out subjects across baseline and classifier-based synthesis methods.
- 3. EXPERIMENTS: 6 of 7 synthesis registrations were qualitatively useful, compared with 4 of 7 baseline registrations.Results were visually inspected using cross correlation.
- 3. EXPERIMENTS: The landmark-error analysis compares two training subjects with four held-out subjects across baseline and two classifier-based synthesis approaches.Each synthesis approach is evaluated with three cost functions, alongside inter-human landmark-placement error.
- 3. EXPERIMENTS: Catastrophic landmark-transfer errors occurred more often with the baseline than with any synthesized image.
4. DISCUSSION
Mutual information underperformed cross correlation in both baseline and synthesis experiments, while learned synthesis improved registration robustness. The method generalized from baseline-successful subjects to some more challenging subjects, although its synthesized images did not visually resemble the target modality.
- Mutual information underperformed cross correlation for both baseline and synthesis experiments.The authors suggest the acquired images’ small dynamic range may contribute to this result.
- The classifier was trained only on subjects where baseline registration succeeded but generalized sufficiently to register some subjects on which baseline failed.
- The synthesized images did not resemble the target modality to human observers, despite being more similar to registration cost functions.
- Learned synthesis produced more robust registration by reducing unusable subject registrations.