PaperlayerView exact revision on arXiv

Source-linked AI summary

Photometric Supernova Classification With Machine Learning

Michelle Lochner, Jason D. McEwen, Hiranya V. Peiris, Ofer Lahav, Max K. Winter

arXiv:1603.00882v3astro-ph.IMastro-ph.CO

TL;DR

The paper addresses how to classify supernovae photometrically when upcoming surveys make spectroscopic confirmation of every object impossible. It combines multiple light-curve feature representations with machine-learning classifiers and finds strong classification without redshift information when training is representative.

  • Problem

    Upcoming imaging surveys will discover more supernovae than can receive spectroscopic confirmation, creating a need for automated photometric type classification.

  • Method

    The pipeline extracts features from simulated DES light curves using several approaches and classifies them with machine-learning algorithms, comparing representative and non-representative training sets.

  • Results

    SALT2 fits and wavelet features with boosted decision trees each achieved an AUC of 0.98.

  • Takeaways & Limitations

    With boosted decision trees and a representative training set, classification performance did not improve when redshift information was added.

  • Takeaways & Limitations

    The paper notes that non-representative training sets can require transfer-learning techniques to mitigate sample-selection bias, which it does not study in depth.

Abstract

from arXiv · show

Automated photometric supernova classification has become an active area of research in recent years in light of current and upcoming imaging surveys such as the Dark Energy Survey (DES) and the Large Synoptic Survey Telescope, given that spectroscopic confirmation of type for all supernovae discovered will be impossible. Here, we develop a multi-faceted classification pipeline, combining existing and new approaches. Our pipeline consists of two stages: extracting descriptive features from the light curves and classification using a machine learning algorithm. Our feature extraction methods vary from model-dependent techniques, namely SALT2 fits, to more independent techniques fitting parametric models to curves, to a completely model-independent wavelet approach. We cover a range of representative machine learning algorithms, including naive Bayes, k-nearest neighbors, support vector machines, artificial neural networks and boosted decision trees (BDTs). We test the pipeline on simulated multi-band DES light curves from the Supernova Photometric Classification Challenge. Using the commonly used area under the curve (AUC) of the Receiver Operating Characteristic as a metric, we find that the SALT2 fits and the wavelet approach, with the BDTs algorithm, each achieves an AUC of 0.98, where 1 represents perfect classification. We find that a representative training set is essential for good classification, whatever the feature set or algorithm, with implications for spectroscopic follow-up. Importantly, we find that by using either the SALT2 or the wavelet feature sets with a BDT algorithm, accurate classification is possible purely from light curve data, without the need for any redshift information.

1. INTRODUCTION

Upcoming wide-field surveys will produce more supernovae than can receive spectroscopic type confirmation, motivating automated photometric classification. The paper therefore develops a multi-faceted pipeline and examines redshift dependence and training-set representativeness.

  • DES and LSST-scale datasets will allow spectroscopic follow-up for only a small fraction of discovered supernovae.
  • Photometric classification supports cosmological, astrophysical, and transient studies by providing type probabilities when spectroscopy is unavailable.
  • The study compares model-dependent and more independent approaches while investigating whether host-galaxy photometric redshift information affects classification.
  • It also evaluates representative and non-representative training sets because spectroscopic samples can be biased toward brighter, lower-redshift objects.
  • The pipeline combines four light-curve feature-extraction methods with five representative machine-learning algorithms.

2. SUPERNOVA SIMULATIONS

The study evaluates photometric supernova classification methods using simulated DES light curves from the Supernova Photometric Classification Challenge. The data include simulated host-galaxy redshifts, whose availability to the classifier is varied.

  • The experiments use simulated DES light curves from the Supernova Photometric Classification Challenge, constructed from non-Ia templates and SALT2 and MLCS2k2 type Ia models.
  • The dataset provides simulated photometric host-galaxy redshifts, allowing the study to test classification with or without this information.
  • The challenge supplied 1103 spectroscopically confirmed training objects and retained 20216 objects for testing.
  • The training set is non-representative in brightness and redshift because spectroscopic follow-up prioritizes bright objects.

3. EXTRACTING USEFUL LIGHT CURVE FEATURES

The pipeline converts multiband supernova light curves into lower-dimensional feature sets for machine learning classification. It compares model-dependent, parametric, and model-independent extraction strategies while evaluating their downstream classification performance.

  • Feature extraction summarizes raw light-curve data into lower-dimensional features that describe the data and separate the classes.
  • A simulated DES type Ia light curve is measured in four griz bands, with data points and error bars compared against a best-fitting SALT2 model.
  • The study compares feature sets by measuring how effectively they support several machine learning algorithms.
  • Deep learning can learn summary features automatically but requires high computing resources and much larger training sets; traditional extraction was adequate here.
  • The feature-extraction approaches range from supernova-template methods through parametric models to a model-independent wavelet approach.
  • The pipeline extracts four feature sets from the same data and applies five different machine learning algorithms.

3.1. Visualizing Feature Sets with t-distributed Stochastic Neighbor Embedding (t-SNE)

The section introduces t-SNE as a visualization method for assessing whether high-dimensional feature sets separate supernova classes. It reduces the feature space while preserving pairwise similarity relationships as much as possible.

  • The technique helps assess class separation when feature sets remain too large to inspect directly after dimensionality reduction.
  • t-SNE produces a low-dimensional representation of high-dimensional feature sets while clustering similar features together.
  • t-SNE first estimates pairwise similarity probabilities from Euclidean distances, then seeks low-dimensional values that preserve those relationships.

3.2. Template Fitting Approach to Classification

The template-fitting approach uses SALT2, a commonly used type Ia light-curve model, to derive features for photometric supernova classification. Its low-dimensional feature space shows class structure, although type Ia and Ibc remain difficult to distinguish.

  • SALT2 is selected as the most commonly used model for type Ia supernovae.
  • The SALT2 flux model uses phase, wavelength, an average spectral sequence, higher-order variability components, and a color-correction law.
  • The SALT2 feature set contains redshift, time of peak brightness, normalization, a variability parameter, and color.
  • The SALT2 feature visualizations show fairly separated classes, especially with redshift, but type Ia and Ibc remain difficult to distinguish.

3.3. General Parametric Approach to Classification

The general parametric approach replaces supernova-specific templates with fitted light-curve parameterizations, including models that can represent different tail behavior and double peaks. Model 1 and Model 2 features do not appear well-separated in t-SNE visualizations, while a high-parameter alternative was less robust in fitting and classification.

  • 3.3. General Parametric Approach to Classification: The general parametric approach uses fitted parameters from two published light-curve models instead of relying on extensive template knowledge.
  • 3.3. General Parametric Approach to Classification: The two models differ in tail treatment, ability to describe double peaks, and parameter count.
  • 3.3. General Parametric Approach to Classification: The linear piecewise alternative was less robust because it used 11 parameters per filter and typically produced poor classification performance.
  • 3.3.1. Model 1: Model 1 describes each-band flux with a spline-based light curve and a tail function that keeps the flux finite as time approaches infinity.
  • 3.3.1. Model 1: Model 1 produces 20 fitted features across four bands, or 21 when redshift is included.
  • 3.3.1. Model 1: Model 1 features do not appear easily separable in t-SNE visualizations with or without redshift.
  • 3.3.2. Model 2: Model 2 extends Model 1 to allow double-peaked light curves and produces 24 features, or 25 including redshift.
  • 3.3.2. Model 2: Model 2 features also do not appear well-separated in t-SNE visualizations.

3.4. Wavelet decomposition approach to classification

The wavelet approach provides a model-independent complement to model-dependent light-curve features. It interpolates curves, decomposes them with wavelets, and uses PCA to obtain compact features for classification.

  • Motivation: Model-dependent feature extraction can be sensitive to mis-specified light-curve models, motivating a complementary model-independent approach.Simple models may fail to capture important light-curve characteristics efficiently.
  • Wavelet features: Wavelet coefficients can remain similar for supernovae with different explosion times or redshifts because suitable transforms are approximately translation- and stretch-invariant.This makes wavelets a natural choice for comparing differently aligned light curves.
  • Wavelet features: The wavelet pipeline interpolates light curves with Gaussian processes, performs a redundant wavelet decomposition, and reduces the resulting features with PCA.Gaussian-process interpolation incorporates uncertainty information and produces less biased interpolated values than spline interpolation.
  • Wavelet features: The adopted implementation uses the a trous transform with symlet wavelets, and its results were not highly dependent on the wavelet family chosen.The a trous transform provides dyadic scale-invariance; alternative constructions remain possible.
  • Dimensionality reduction: The decomposition produces 1600 coefficients per object, which PCA reduces to 20 dimensions while retaining 98% of the dataset information.The high-dimensional redundancy preserves translation invariance before dimensionality reduction.
  • Feature-space assessment: t-SNE plots show encouraging separation between classes, including some separation between type Ia and Ibc, suggesting strong classification performance from wavelet features.The separation is presented as an expectation before the classification results.

4. MACHINE LEARNING FOR CLASSIFICATION

The paper compares five representative supervised machine-learning algorithms for classifying supernovae from extracted light-curve features. These methods differ in their modeling assumptions, flexibility, computational demands, and sensitivity to training data or tuning.

  • Algorithms: The study considers naive Bayes, k-nearest neighbors, multilayer perceptrons, support vector machines, and boosted decision trees.These algorithms represent probabilistic, neighbor-based, neural-network, margin-based, and ensemble approaches.
  • Algorithm properties: Naive Bayes is fast and scales well to high dimensions but assumes independent, Gaussian-distributed features.The paper notes that one or both assumptions are frequently violated.
  • Algorithm properties: KNN classifies objects by a weighted vote among nearby training examples but is computationally intensive for large datasets and training-set dependent.The method uses Euclidean distances with weights inversely proportional to distance.
  • Algorithm properties: The multilayer perceptron learns nonlinear mappings from input features to output classes, while its performance is sensitive to hyperparameter tuning.The implementation uses layered neurons with nonlinear activation functions and backpropagation.
  • Algorithm properties: SVMs separate classes with a feature-space hyperplane and can model nonlinear relationships through kernels, but probability estimation with Platt scaling is computationally intensive.The paper describes SVMs as versatile and effective even in high dimensions.
  • Algorithm properties: Boosted decision trees combine many decision trees to produce ensemble classifications and are described as generally excellent classifiers.Ensembling addresses the poor generalization of individual complex decision trees.

5. PHOTOMETRIC SUPERNOVA CLASSIFICATION PIPELINE

The classification pipeline systematically combines feature extraction, machine-learning algorithms, training-set design, and redshift availability. Performance is compared across representative and non-representative training sets, with and without host-galaxy photometric redshifts.

  • Pipeline design: For each feature set and machine-learning algorithm, the study splits the data into training and test sets and evaluates representative and non-representative training separately.The representative training sets are randomly selected with the same size as the challenge training set.
  • Training sets: Representative training sets contain 1103 randomly selected objects, matching the challenge training-set size, and are repeated to measure result variation.The non-representative set is the original SPCC training set.
  • Redshift information: The pipeline runs all four feature-extraction methods both with and without photometric host-galaxy redshift information.This tests whether redshift is essential for classification given the difficulty of obtaining accurate photometric redshifts.
  • Evaluation: Features are standardized, hyperparameters are selected by cross-validation, and the resulting classifiers are compared using ROC curves.Standard scaling centers features at zero and divides by their standard deviations.

6. RESULTS

Across the tested feature sets and algorithms, BDTs perform best, while SALT2 and wavelet features provide the strongest classification. Performance depends strongly on representative training data, and BDTs with SALT2 or wavelet features can classify without redshift information.

  • BDTs outperform the other tested algorithms across the feature-extraction methods, while SALT2 and wavelet features outperform the parametric models.The comparison includes naive Bayes, KNN, SVMs, ANNs, and BDTs.
  • Type II supernovae are classified accurately by all feature-extraction methods, whereas type Ibc supernovae are substantially more difficult to classify.The difficulty for type Ibc objects reflects their similarity to type Ia light curves and their relatively small representation in the dataset.
  • SALT2 performs significantly better than the other methods at identifying type Ibc supernovae.
  • At a 90% purity threshold for type Ia objects, completeness is 85%, 41%, 7.8%, and 83% for SALT2, Model 1, Model 2, and wavelets, respectively.At a 90% completeness threshold, the corresponding purities are 87%, 77%, 75%, and 87%.
  • With BDTs, redshift information is not essential because the algorithm can differentiate supernova classes without it, especially for SALT2 and wavelet features.Redshift helps SALT2 features more than wavelet features, but the best-performing algorithm removes the need for redshift under a representative training set.
  • All feature sets and algorithms perform significantly worse with a non-representative training set, making representative coverage critical for classification.The authors therefore favor fewer, fainter objects of all types over larger samples dominated by brighter objects when spectroscopic follow-up is limited.

7. CONCLUSIONS

SALT2 fits and wavelet decomposition achieved excellent photometric supernova classification, while representative training data proved crucial. Classification can remain accurate without redshift information, although further improvements are possible.

  • Performance: 0.98 AUC was achieved by both SALT2 fits and wavelet decomposition, outperforming parametric fits.Boosted decision trees performed best among the five evaluated machine learning algorithms.
  • Performance: About 90% purity and 84% completeness were achieved by both SALT2 and wavelet methods.Purity can be increased by sacrificing completeness.
  • Redshift information: With boosted decision trees and a representative training set, redshift information did not improve classification performance.A well-classified supernova subset can therefore be obtained from light curves without redshift information.
  • Training data: Representative training data was crucial for good performance across all feature extraction methods and algorithms.The training set contained around 5% of the dataset, motivating spectroscopic follow-up of fainter objects of all types.
  • Future work: The framework enables direct comparison of future feature extraction methods and machine learning algorithms.The authors also plan applications to SDSS data and LSST observing-strategy studies.

APPENDIX HYPERPARAMETER SELECTION

Hyperparameters were selected by cross-validation to maximize AUC, with values varying slightly according to the training set. The appendix table presents an example configuration when redshift is included with representative data.

  • Selection: Hyperparameters were selected using cross-validation to maximize AUC.The selected values vary slightly depending on the training set used.
  • Configuration: Table 5 lists hyperparameters for each machine learning algorithm and feature set.The example case includes redshift and uses a representative dataset.
Loading 1603.00882v3…