Source-linked AI summary
Using transfer learning to detect galaxy mergers
Sandro Ackermann, Kevin Schawinski, Ce Zhang, Anna K. Weigel, M. Dennis Turp
TL;DR
Some merger images lack two distinct luminosity peaks, and this paper tests deep CNNs with ImageNet-pretrained weights for automatic merger detection. The CNNs outperform prior automatic methods, while transfer learning modestly improves generalization for the tested dataset sizes.
Problem
Some merger images may lack two distinct luminosity peaks, making such peaks an incomplete basis for visual merger detection.
Source: §1 · p. 1Method
The paper trains deep CNNs for galaxy-merger classification, initializing them with ImageNet-pretrained weights to test transfer learning.
Source: §2.2 · p. 3Results
CNNs significantly outperform prior automatic merger-detection methods, while transfer learning modestly improves generalization for the tested dataset sizes.
Source: §6 · p. 9Takeaways & Limitations
The resulting merger sample agrees with the Darg et al. (2010) catalogue in colour-mass distribution and stellar mass function.
Source: §6 · p. 9Applying a classifier to a dataset with different underlying statistical properties provides no guarantee of a sensible merger sample.
Source: §5 · p. 7
Abstract
from arXiv · showhide
We investigate the use of deep convolutional neural networks (deep CNNs) for automatic visual detection of galaxy mergers. Moreover, we investigate the use of transfer learning in conjunction with CNNs, by retraining networks first trained on pictures of everyday objects. We test the hypothesis that transfer learning is useful for improving classification performance for small training sets. This would make transfer learning useful for finding rare objects in astronomical imaging datasets. We find that these deep learning methods perform significantly better than current state-of-the-art merger detection methods based on nonparametric systems like CAS and GM$_{20}$. Our method is end-to-end and robust to image noise and distortions; it can be applied directly without image preprocessing. We also find that transfer learning can act as a regulariser in some cases, leading to better overall classification accuracy ($p = 0.02$). Transfer learning on our full training set leads to a lowered error rate from 0.038 $\pm$ 1 down to 0.032 $\pm$ 1, a relative improvement of 15%. Finally, we perform a basic sanity-check by creating a merger sample with our method, and comparing with an already existing, manually created merger catalogue in terms of colour-mass distribution and stellar mass function.
1 INTRODUCTION
The paper investigates deep CNNs for automatic galaxy-merger detection, focusing on feature learning from images and potential improvements over existing automatic methods. Its motivation is that prior approaches have limitations in detecting merger diversity, while CNNs can learn task-specific features directly from pixels.
Introduction
- Close-pairs detection can be hindered by imprecise radial distances and heuristic target identification, and some mergers lack two distinct luminosity peaks (Lin et al. (2004); Woods & Geller (2007)).The approach uses luminosity peaks to select spectroscopic targets and redshifts to assess whether pairs are interacting.Source: §1 · p. 1
- Hand-crafted CAS and GM20 features can struggle to capture the diversity of merger appearances, with detection sensitivity varying across merger stages (Conselice (2003); Lotz et al. (2004); Lotz et al. (2008); Lotz et al. (2010)).These methods classify galaxies using prescribed image features such as concentration, asymmetry, clumpiness, Gini coefficient, and second-order flux moments.Source: §1 · pp. 1–2
- Galaxy Zoo 1 crowdsourced visual classifications of about one million SDSS galaxies, and its merger classifications serve as the study’s ground truth for training the classifier (Lintott et al. (2008, 2010)).Source: §1 · p. 2
- The study tests whether state-of-the-art CNN architectures can improve automatic galaxy-merger detection, building on their success in image classification and galaxy-imaging tasks.The authors identify this evaluation as the paper’s main focus.Source: §1 · p. 2
- CNNs discover task-specific features from raw pixels without hand-crafted detectors, preprocessing, or dimensionality reduction; the authors describe this as the first use of feature learning for visual galaxy-merger detection.They state that CNNs are robust to noise and image defects when these are represented in the training data.Source: §1 · p. 2
2 METHOD
The method evaluates Xception CNNs for galaxy-merger classification, testing ImageNet transfer learning as a proposed regularizer for small-sample training. It uses SDSS and Galaxy Zoo-derived labels and compares CNN performance with prior methods and random initialization across restricted training sizes.
2.1 Deep Convolutional Neural Networks
- The method uses Xception, whose convolutional layers apply shared local filters, yielding translational invariance and fewer learnable parameters; training minimizes a loss, followed by inference with fixed weights.Convolutional layers’ reduced parameter count also acts as a regularizer.Source: §2.1 · pp. 2–3
2.2 Transfer Learning
- ImageNet-pretrained CNN weights are tested as a regularizer to reduce overfitting and improve generalization over random initialization when classifying rare astronomical objects.The authors motivate transfer learning by the risk that large networks fit noise in small training sets.Source: §2.2 · p. 3
2.3 Dataset
- Training data comprise 3,003 catalogue mergers and 10,000 randomly selected non-interacting Galaxy Zoo galaxies from SDSS DR7, with stratified mini-batches balancing the classes.Merger labels use the Darg et al. (2010) catalogue, based on Galaxy Zoo weighted-merger-vote fractions and expert visual confirmation; the non-interacting sample has fm < 0.2.Source: §2.3 · pp. 3–4
2.4 Experiment
- The experiments compare a modern CNN with previous merger-classification methods and test transfer learning against random initialization using training sets of 3,000, 1,500, 900, 500, and 300 images.The transfer-learning comparison is repeated across the restricted training-set sizes to assess how training-set size affects its utility.Source: §2.4 · p. 4
3 RESULTS
Transfer learning significantly outperformed random initialization overall across five training-set sizes, although the evidence varied by size. The classifier assigned binary labels using a 0.5 merger-probability threshold on held-out test images.
3 RESULTS
- The classifier used a merger-probability threshold of 0.5 to assign binary labels to images in the held-out test set.Its output ranged from 0 for non-interacting galaxies to 1 for merger systems.Source: §3 · p. 4
- Across five training-set sizes, transfer learning had a significant overall advantage over random initialization (combined p = 0.02).The figure shows test-error comparisons for training sets of 3000, 1500, 900, 500, and 300 images.Source: Figure 2 · p. 4, §3 · p. 4
- Transfer learning was significant at sizes 300 (p = 0.05) and 3000 (p = 0.03), plausible at 1500 (p = 0.12), and inconclusive at 900 (p = 0.32) and 500 (p = 0.72).Significance was assessed using a one-tailed Welch’s t-test on test-error samples from four-fold cross-validation.Source: §3 · p. 4, Figure 2 · p. 4
4 QUANTIFYING PERFORMANCE
The classifier usually assigns merger probabilities near 0 or 1, and its ROC performance is near-perfect. Isotonic recalibration improves probability calibration, while errors often involve cases that are difficult for humans to classify.
4.1 Histogram and calibration plot
- The CNN’s merger probabilities cluster near 0 or 1, and bins of predictions are somewhat consistent with their observed merger fractions.Unclear classifications are rare; the calibration plot compares predicted probability with the fraction of true mergers in each bin.Source: §4.1 · pp. 4–5, Figure 3 · p. 5, Figure 4 · p. 5
4.2 Recalibration
- Isotonic regression fitted on one third of the test set improves calibration, with the remaining two thirds used to evaluate it.The method fits a monotonically non-decreasing function mapping classifier outputs to approximate true probabilities; afterward, the plot is very consistent with the diagonal and the histogram has more middle probabilities.Source: §4.2 · p. 5, Figure 5 · p. 5, Figure 6 · p. 6
4.3 ROC (Receiver Operating Characteristic) curve
- AUROC = 0.9922, indicating near-perfect discrimination without fixing a threshold and performance invariant to the test set’s merger fraction.The ROC curve is consistently and significantly above the random-classifier diagonal.Source: Figure 7 · p. 6, §4.3 · p. 6
4.4 Failure modes
- Misclassifications often involve potential galaxy overlaps and very early- or late-stage mergers, which can also be hard for human classifiers.The classifier handles diverse merger appearances and appears able to identify star overlaps correctly.Source: §4.4 · p. 6
5 PROPERTIES OF THE MERGER SAMPLE
The transfer-learning CNN produced a merger sample whose colour and mass-function distributions broadly resemble those of visually classified mergers, with differences in colour, space density, and completeness. The comparison is a post-hoc check of the sample, not a guarantee that applying the classifier to a dataset with different statistical properties will yield a sensible catalogue.
5 PROPERTIES OF THE MERGER SAMPLE
- The authors applied the largest transfer-learning model to Galaxy Zoo I objects in the Darg et al. (2010) redshift range; pm>0.95 selected 7,980 of 328,151 objects.Four-fold cross-validation ensured that at least one classifier had not seen each object during training, enabling pm estimates without inference on that object's training or validation exposure.Source: §5 · p. 7, §5.1 · p. 8
5.2 Stellar mass functions
- log(M*/M⊙)=10.81±0.04, log(Φ*/h^3 Mpc−3)=−3.43±0.02, and α=−0.62±0.09; CNN and visually classified merger mass-function shapes resemble, but CNN Φ* is higher.The CNN fit is compared with the visually classified major-merger mass function of Weigel et al. (2017b).Source: §5.2 · pp. 8–9
- The higher Φ* does not establish more CNN-detected mergers per volume: the samples differ in mass-ratio cuts, mass assignments, and counting of observed partners.Weigel et al. (2017b) restricts to major mergers and counts one mass per system; the CNN sample includes all spectroscopically observed galaxies above the probability cut and can count both partners.Source: §5.2 · pp. 8–9
5.1 Colour-Mass diagram
- CNN mergers span the blue cloud to red sequence like Darg et al. (2010), but are redder; the difference is significant only at 9 < log(M/M⊙) ≤10 (p=0.009).The colour-mass comparison uses a volume-complete sample at 0.02 ≤ z ≤ 0.06; the stated significance is from the Kolmogorov-Smirnov test.Source: §5.1 · p. 8, Figure 12 · p. 9
5.2 Stellar mass functions
- The CNN sample is complete only to log(M/M⊙)=10, versus 9 for Darg et al. (2010); its redder colours and lower mass-to-light ratios account for the difference.The colour difference was significant in the lowest mass bin, and mass-to-light ratios affect conversion from luminosity completeness to stellar-mass completeness.Source: §5.1 · p. 8, §5.2 · p. 9
6 CONCLUSIONS
State-of-the-art CNNs significantly outperform previous methods for visual galaxy-merger detection; transfer learning modestly improves generalisation at the dataset sizes studied. The method also produces well-calibrated merger probabilities and yields a sample consistent with an existing catalogue’s colour-mass distribution and stellar mass function.
6 CONCLUSIONS
- State-of-the-art CNNs significantly outperform previous methods for automatic visual detection of galaxy mergers.The conclusion identifies this as a significant improvement over prior methods.Source: §6 · p. 9
- For the dataset sizes studied, transfer learning can modestly improve the classifier’s generalisation power.The authors describe initialisation with ImageNet weights as one example.Source: §6 · p. 9
- The method’s merger sample agrees with the Darg et al. (2010) catalogue in colour-mass distribution and stellar mass function.Figure 13 presents a stellar-mass-function comparison for transfer-learning detections, visually selected major mergers, and all galaxies.Source: §6 · p. 9, Figure 13 · p. 10
- The method can also detect rare astronomical objects, including gravitational lenses (Marshall et al. 2016), galaxies with shocked interstellar medium (Alatalo et al. 2016), AGN ionization echoes (Keel et al. 2012), and ring galaxies (Buta 1995).The authors state that their methods are not specific to merger classification.Source: §6 · p. 9
- For each image, the classifier outputs pm ∈ [0, 1], interpretable as the true probability that the object belongs to a merger system.The authors describe these classifications as well-calibrated.Source: §6 · p. 9
APPENDIX A: CS PROTOCOL · A1 Main Experiment
The A1 protocol tests whether transfer-learning CNNs can outperform state-of-the-art merger classifiers, using cross-validation and classification and ROC metrics. The supplied passages state this as a hypothesis and expected result, not a measured outcome.
A1 Main Experiment
- At classification threshold p = 0.5, the evaluation measures precision, recall, and F-1 score, and also evaluates the ROC curve and AUROC.Source: A1 Main Experiment · p. 11
- The protocol uses K-fold cross-validation, with separate validation and test folds, and reports test-set quality scores across K iterations.The final ImageNet-trained fully connected layers are replaced with two randomly initialized outputs; only those new weights are trained for 40 epochs while the remaining layers are frozen, using SGD with momentum 0.9, learning rate 0.000015, and validation-accuracy early stopping.Source: A1 Main Experiment · p. 11
- The expected result is that transfer-learning CNNs outperform state-of-the-art merger-classification methods on the chosen metrics.This is stated as a hypothesis and expected result; the supplied passages do not report measured performance.Source: A1 Main Experiment · p. 11
A2 Lesion Study: The Impact of Transfer Learning
The lesion study tests whether transfer learning outperforms random initialization, particularly with smaller training sets, using test error as the comparison metric. The protocol compares both approaches across five dataset sizes with four-fold cross-validation.
A2 Lesion Study: The Impact of Transfer Learning
- The study tests whether transfer learning yields lower test error than deep learning with random initialization, especially for smaller training sets.Test error is the proxy for comparing the approaches; this is the study’s hypothesis, not a reported result.Source: A2 Lesion Study: The Impact of… · p. 11
- Five dataset subsets—10%, 17%, 30%, 50%, and 100% of the full dataset—are each evaluated with both approaches using K-fold cross-validation, for ten experiments total.The study uses K = 4, chosen with data availability and computational resources in mind.Source: A2 Lesion Study: The Impact of… · p. 11
- For random initialization, the network weights are randomly initialized and trained with SGD, momentum 0.9, and learning rate 0.000015, using validation accuracy for early stopping.Test-set error scores are collected from the cross-validation loop.Source: A2 Lesion Study: The Impact of… · p. 11