Rotation-invariant convolutional neural networks for galaxy morphology prediction

Sander Dieleman; Kyle W. Willett; Joni Dambre

Source-linked AI summary

Rotation-invariant convolutional neural networks for galaxy morphology prediction

Sander Dieleman, Kyle W. Willett, Joni Dambre

arXiv:1503.07077v1Mar 24, 2015astro-ph.IMastro-ph.GAcs.CVcs.LGcs.NEstat.ML

TL;DR

Galaxy surveys produce more images than experts or crowdsourcing can feasibly annotate, motivating automated morphology prediction. The paper develops a convolutional neural network that exploits translational and rotational symmetry, achieving near-perfect agreement on high-consensus cases and enabling confidence-based filtering for expert review.

Problem
Large galaxy surveys require accurate morphology classifications, but manual inspection and crowdsourcing do not scale to their growing image collections.
Method
The paper develops a convolutional neural network for galaxy morphology prediction that exploits translational and rotational symmetry in galaxy images.
Results
Near-perfect accuracy is achieved for most questions on images with high Galaxy Zoo participant agreement, and a single network produces competitive results while the winning solution averaged multiple networks.
Takeaways & Limitations
Confidence-based filtering can reduce the expert workload by forwarding challenging images for manual annotation without affecting accuracy.
Takeaways & Limitations
The study uses approximately 55,000 galaxy images, described as small by modern standards, leaving larger annotated datasets as an opportunity to improve predictive performance.

Abstract

from arXiv · show

Measuring the morphological parameters of galaxies is a key requirement for studying their formation and evolution. Surveys such as the Sloan Digital Sky Survey (SDSS) have resulted in the availability of very large collections of images, which have permitted population-wide analyses of galaxy morphology. Morphological analysis has traditionally been carried out mostly via visual inspection by trained experts, which is time-consuming and does not scale to large ($\gtrsim10^4$) numbers of images. Although attempts have been made to build automated classification systems, these have not been able to achieve the desired level of accuracy. The Galaxy Zoo project successfully applied a crowdsourcing strategy, inviting online users to classify images by answering a series of questions. Unfortunately, even this approach does not scale well enough to keep up with the increasing availability of galaxy images. We present a deep neural network model for galaxy morphology classification which exploits translational and rotational symmetry. It was developed in the context of the Galaxy Challenge, an international competition to build the best model for morphology classification based on annotated images from the Galaxy Zoo project. For images with high agreement among the Galaxy Zoo participants, our model is able to reproduce their consensus with near-perfect accuracy ($> 99\%$) for most questions. Confident model predictions are highly accurate, which makes the model suitable for filtering large collections of images and forwarding challenging images to experts for manual annotation. This approach greatly reduces the experts' workload without affecting accuracy. The application of these algorithms to larger sets of training data will be critical for analysing results from future surveys such as the LSST.

1 INTRODUCTION

Large galaxy surveys require accurate morphology classifications at a scale that manual inspection, existing automated systems, and crowdsourcing cannot sustain. The paper proposes a symmetry-aware neural network to automate this task and reduce annotation effort.

Large surveys provide millions of galaxy images, but manually annotating their morphologies is impractical for individual astronomers or small teams.
Existing automated classifiers struggled to reach the reliability required for scientific analysis, while Galaxy Zoo crowdsourcing does not scale indefinitely with growing data volumes.
Deep neural networks became more feasible because of advances in image classification, computing power, and the availability of reliably annotated Galaxy Zoo images.
Automated classifications are presented as a necessary complement to expert and crowdsourced catalogs as future surveys increase image volumes.
The proposed convolutional neural network exploits translational and rotational symmetry and learns increasingly abstract image representations from Galaxy Zoo 2 annotations.
The model won the Galaxy Challenge against 326 participants and automatically annotates galaxy-image catalogs with morphology information.

2 GALAXY ZOO

Galaxy Zoo uses online crowdsourcing to collect detailed galaxy-morphology classifications through a branching decision tree. Aggregated participant votes provide reliable morphology information used in research and model training.

Galaxy Zoo asks online users to describe galaxy morphology from color images using questions whose answers determine subsequent questions.
The Galaxy Zoo 2 scheme contains 11 questions and 37 answers spanning Hubble-tuning-fork categories and more irregular morphologies.
Because of the decision-tree structure, each participant answers only a subset of questions for each galaxy classification.
Aggregated answers form weighted vote fractions for the decision tree, which estimate confidence levels for each answer.
Galaxy Zoo data have supported studies of galaxy structure, formation, and evolution, while comparisons show strong agreement with expert and automated classifications.

3 THE GALAXY CHALLENGE

The Galaxy Challenge evaluated models that predict detailed Galaxy Zoo 2 morphology probabilities from galaxy images. Its setup reflects a branching annotation scheme and scores predictions against crowdsourced probability targets.

Galaxy Zoo vote fractions were post-processed for reliability, including user-consistency weighting and compensation for apparent-magnitude and image-size biases.
The competition used Galaxy Zoo 2 images and morphology data to develop models applicable to varied galaxy images in future surveys.
The final training set contained 61,578 color galaxy images with probabilities for 37 decision-tree answers, alongside a 79,975-image evaluation set without morphology labels.
Competition models predicted answer probabilities as a regression problem and were evaluated using root-mean-square error against crowdsourced probabilities.
The RMSE metric emphasizes questions with higher answer probabilities, particularly those higher in the decision tree.
The decision tree asks Question 1 for every image, while later questions depend on previous answers and correspond to broader morphological categories.
The provided probabilities are noisy and biased, so trained models may reproduce crowd classification behavior rather than true galaxy morphology.

4 RELATED WORK

Earlier galaxy-morphology methods used handcrafted features, small class sets, or limited datasets. This work instead applies convolutional networks to raw pixels for fine-grained classification and exploits rotational symmetry.

Earlier methods often used much smaller datasets and networks with 10^1–10^3 trainable parameters, whereas modern architectures can handle at least approximately 10^7 parameters.
Prior approaches included general-purpose WND-CHARM features with nearest-neighbor classifiers, PCA with neural networks, and kernel SVMs on raw pixels.
The proposed approach works directly from raw pixels and learns task-specific hierarchical features instead of relying on handcrafted feature extraction and selection.
Unlike many earlier studies limited to 2–5 galaxy classes or scalar morphology values, Galaxy Zoo classifications are more fine-grained and challenging.
The model exploits rotational symmetry in galaxy images, extending convolutional approaches beyond their usual use of translational symmetry.

5 BACKGROUND

Deep learning builds hierarchical representations through nonlinear layers optimized from data. Convolutional and pooling layers exploit image structure using local connectivity, shared filters, and dimensionality reduction.

Deep learning: Deep learning models form progressively abstract representations through multiple nonlinear processing layers whose parameters are learned from data.Hidden layers learn task-useful representations while the output layer predicts desired quantities.
Deep learning: A feed-forward layer transforms its input with weights, biases, and an elementwise activation function, producing the next representation.The network maps input x_0 to output x_N, representing one or more predictions.
Convolutional neural networks: Convolutional layers replace dense matrix products with local convolutions, sharing feature detectors across the input to exploit translational symmetry.Shared detectors can identify features such as oriented edges throughout an image.
Convolutional neural networks: Convolutional layers use far fewer parameters than dense layers, which can improve generalization and support larger input dimensionalities.The reduction follows from restricted connectivity and shared filters.
Convolutional neural networks: Pooling layers aggregate local feature-map regions to reduce dimensionality, enlarge higher layers’ context, and provide invariance to small translations.Pooling typically uses maximum or mean aggregation and has no trainable parameters.

6 EXPLOITING ROTATIONAL SYMMETRY

The method exploits the fact that rotating a galaxy image should not change its morphological classification by processing multiple rotated and optionally flipped viewpoints with shared convolutional layers. Their representations are combined for prediction, while rotation introduces interpolation and alignment complications.

Rotational symmetry: Galaxy morphology is treated as rotationally symmetric because rotating an image should not affect its morphological classification.The model applies the same feature detectors to multiple rotated versions, increasing parameter sharing with a positive effect on generalization performance.
Challenges: Non-90° rotations require interpolation and produce edges misaligned with the pixel grid, making rotational symmetry harder to exploit than translational symmetry.The method must explicitly instantiate multiple rotated image versions.
Assumptions: Galaxy Zoo 2 probabilities contain no detected handedness-related rotational or translational biases, although such biases would undermine the rotational-invariance assumption.This consequence applies if those biases exist.
Architecture: The approach creates rotated and flipped viewpoints, processes each with the same convolutional-pooling network, and concatenates their representations before dense prediction layers.Dense layers aggregate high-level features extracted from different viewpoints.
Architecture: Viewpoints are cropped around the galaxy centre to reduce redundancy, input size, and computation time while retaining an informative image region.This is part of the practical viewpoint-extraction procedure.

7 APPROACH

The approach combines preprocessing, augmentation, viewpoint extraction, convolutional processing, and model averaging to predict galaxy morphology probabilities. Its design is driven largely by overfitting risk in a model with millions of parameters trained on roughly 5×10^4 images, while colour information and diverse model errors support performance.

Pipeline: The processing pipeline has five stages: input preprocessing, augmentation, viewpoint extraction, a convolutional neural network, and model averaging.These stages obtain answer probabilities from an image.
Experimental setup: The competition provided 61,578 training images with answer probabilities and 79,975 evaluation images, with feedback scored by RMSE on about 25% of evaluation images.The scored subset was not revealed.
Avoiding overfitting: Millions of learnable parameters trained on only 5×10^4 images create a high risk of overfitting, motivating data augmentation, regularization, parameter sharing, and model averaging.These strategies were used to reduce memorization and improve generalization.
Preprocessing: Images were cropped and downscaled to reduce input dimensionality and accelerate training, with little to no effect on predictive performance.Cropping focuses on the centred galaxy.
Preprocessing: Object-centering and rescaling had no significant RMSE effect, but models trained with this normalization made different mistakes, aiding model averaging.Comparable performance with diverse errors is desirable when averaging models.
Preprocessing: Keeping the original colour channels considerably improved predictive performance because artificial colours remained correlated with morphology.The input images were 424 × 424 pixel colour JPEGs.

7.4 Data augmentation

The model uses randomized augmentation and symmetry-aware viewpoint extraction to expand training variation while limiting computational and image-fidelity costs. Its convolutional architecture processes shared representations from multiple viewpoints and produces constrained answer probabilities.

Data augmentation: Random rotation, translation, scaling, flipping, and brightness adjustment augment each training image.Images are perturbed on demand, so the model never sees the exact same training example twice.
Viewpoint extraction: Sixteen viewpoints arise from two crops, horizontal flips, and four rotated corner patches per crop.The crops are 69 × 69 pixels, while the corner patches are 45 × 45 pixels.
Viewpoint extraction: Two affine transformations generate all 16 viewpoints without interpolation, minimizing computation, padding effects, and image-fidelity loss.The resulting viewpoints are processed as 45 by 45 by 3 RGB arrays by the same convolutional architecture.
Network architecture: Four convolutional layers with square filters and pooling feed concatenated viewpoint features into three dense layers.The selected architecture has roughly 42 million trainable parameters and was chosen after evaluating more than 100 architectures.
Output processing: Rectification followed by per-question normalization produces answer distributions, while differentiable post-processing enforces decision-tree probability constraints.This constraint-aware output processing produced a small but significant performance improvement.

7.7 Training

The networks were trained with minibatch gradient descent, Nesterov momentum, scheduled learning-rate reductions, carefully initialized parameters, and dropout to control overfitting.

Optimization: Minibatch gradient descent used batches of 16 and Nesterov momentum with coefficient µ = 0.9.The chosen batch size corresponds to an effective batch size of 256 because each input is processed through 16 viewpoints.
Optimization: Approximately 1.5 million gradient updates covered 25 million training examples, with the learning rate reduced tenfold twice.The schedule changed η from 0.04 to 0.004 after 18 million examples and to 0.0004 after 23 million.
Initialization: Weights were initialized from zero-mean normal distributions with manually selected layerwise variances and positive biases.The initialization was chosen to support gradient flow and reduce saturation risk.
Regularization: Dropout was applied in all three dense layers and was essential for reducing overfitting to manageable levels.

7.8 Model averaging

Prediction averaging combines multiple models and transformed views to improve accuracy, while the authors also note that the unaveraged model performs well enough for fast inference.

Averaging strategy: Model averaging is effective when individual models have similar accuracy and uncorrelated prediction errors.
Transformation averaging: Each model produced predictions for 60 affine transformations spanning 10 rotations, 3 rescalings, and optional horizontal flipping.The predictions were combined with an unweighted average.
Model ensemble: Seventeen model variants were averaged across the 60 transformations, yielding 1020 sets of predictions in total.

8 RESULTS

The evaluation compares prediction averaging and classification agreement with Galaxy Zoo annotations. Performance is strongest for common, high-agreement morphologies and confident predictions, while rare or difficult categories remain challenging.

Competition results: Averaging across 60 transformations and 17 network variants both contributed significantly to the final RMSE score.Table 3 compares the best network without averaging, transformation averaging, and averaging across all 17 variants.
Classification accuracy: Classification accuracy tapers as Galaxy Zoo participant agreement decreases, while high-agreement questions reach near-perfect accuracy for most questions.
Confidence: Highly confident predictions are near-perfect for most questions, enabling automatic filtering and expert review of challenging images.Confidence can therefore identify predictions to trust and cases to defer to experts.
Question-specific performance: Questions 1, 2, 3, 6, and 7 support confident predictions for most examples, whereas questions 5 and 10 generally require substantial manual input.The easier cases include smoothness and roundedness; bulge dominance and spiral-arm tightness remain difficult.
Rare morphologies: The model has more difficulty with infrequent morphology types, and rare answers such as lens or arc were never predicted in the evaluated examples.Recall is generally affected more strongly than precision, indicating conservative prediction of rare morphologies.

9 ANALYSIS

The analysis examines learned filters, layer activations, hidden-unit selectivity, and difficult examples to characterize what the network represents and where it struggles.

Learned representations: 32 first-layer filters respond to colours, patterns, edges, and textures, with edge detectors especially sensitive to curved edges.The curved-edge sensitivity is consistent with the radial symmetry of the input images.
Learned representations: Higher convolutional-layer activations retain input-image geometry, while activations are sparse except in the third layer.The fourth layer is especially sparse, and the third layer lacks a directly following pooling layer.
Learned representations: Topmost hidden units discriminate among galaxy types and exhibit rotation invariance, with some scale invariance and occasional multimodal responses.Maximally and minimally activating examples reveal both sensitivity and discrimination between input types.
Learned representations: One hidden unit detects SDSS dead-pixel artifacts and reproduces Galaxy Zoo participants’ tendency to classify those images as disturbed.The artifact is not a morphological feature of the depicted galaxy.
Error analysis: Larger angular size and non-radially symmetric morphology are the most challenging targets for the model.Difficult examples are also affected by faintness and cropping, whereas smooth, round ellipticals are mostly easiest to classify.

10 CONCLUSION AND FUTURE WORK

The paper concludes that a rotationally symmetry-aware convolutional network can classify galaxy morphology from raw pixels and support large-scale analysis. It identifies larger training sets, broader generalization, and workflow integration as future priorities.

Conclusion: The proposed network predicts fine-grained galaxy morphology from raw Galaxy Zoo 2 pixels while exploiting rotational symmetry.It does not require handcrafted feature extraction.
Conclusion: The rotational-symmetry approach was essential to state-of-the-art performance and won the Galaxy Challenge, while a single network remained competitive.The winning solution averaged predictions from many networks.
Conclusion: The model can be adapted to centered galaxy images and arbitrary morphological decision trees, and its confident predictions are highly reliable for large-scale survey analysis.The implementation uses open-source tools and can be trained and used on consumer hardware.
Future work: The approximately 55,000 galaxy images used here are small by modern standards, motivating training on larger annotated collections.Larger datasets may improve predictive performance and permit greater model capacity without excessive overfitting.
Future work: Generalization across different redshift slices requires care because newer Galaxy Zoo iterations focus on higher-redshift samples.This is an explicit scope boundary for applying the model to newer annotations.
Future work: Future work includes adapting the approach to raw photometric data, other radially symmetric modalities, and deeper network architectures.The authors also identify integration into expert and crowdsourcing annotation workflows as requiring further study.