Source-linked AI summary
Automating Discovery and Classification of Transients and Variable Stars in the Synoptic Survey Era
J. S. Bloom, J. W. Richards, P. E. Nugent, R. M. Quimby, M. M. Kasliwal, D. L. Starr, D. Poznanski, E. O. Ofek, S. B. Cenko, N. R. Butler, S. R. Kulkarni, A. Gal-Yam, N. Law
TL;DR
Synoptic surveys produce more real-time data than astronomers can inspect, motivating automated discovery and classification. The paper develops a machine-learned framework for image-difference discovery and source classification, and demonstrates autonomous operations with rapid follow-up and multiple science results. Its scope is constrained by dependence on bona fide varying sources and unresolved questions about transferring contextual information to new surveys.
Problem
Synoptic survey data volumes exceed the capacity of humans to inspect in real time, while image differences contain many spurious candidates.
Method
The paper combines image-difference discovery with machine-learned classification using expert-trained input and survey-derived contextual information.
Results
The framework operated near real time on real-world data, enabling rapid follow-up and producing 529 autonomously discovered spectroscopically confirmed supernovae plus several notable science results.
Takeaways & Limitations
Autonomous discovery and classification can support initial follow-up without astronomers in the real-time loop and connect survey operations to diverse transient science.
Takeaways & Limitations
The classification results should be interpreted only for sources observed to vary significantly and confirmed as bona fide astrophysical variables or transients.
Abstract
from arXiv · showhide
The rate of image acquisition in modern synoptic imaging surveys has already begun to outpace the feasibility of keeping astronomers in the real-time discovery and classification loop. Here we present the inner workings of a framework, based on machine-learning algorithms, that captures expert training and ground-truth knowledge about the variable and transient sky to automate 1) the process of discovery on image differences and, 2) the generation of preliminary science-type classifications of discovered sources. Since follow-up resources for extracting novel science from fast-changing transients are precious, self-calibrating classification probabilities must be couched in terms of efficiencies for discovery and purity of the samples generated. We estimate the purity and efficiency in identifying real sources with a two-epoch image-difference discovery algorithm for the Palomar Transient Factory (PTF) survey. Once given a source discovery, using machine-learned classification trained on PTF data, we distinguish between transients and variable stars with a 3.8% overall error rate (with 1.7% errors for imaging within the Sloan Digital Sky Survey footprint). At >96% classification efficiency, the samples achieve 90% purity. Initial classifications are shown to rely primarily on context-based features, determined from the data itself and external archival databases. In the ~one year since autonomous operations, this discovery and classification framework has led to several significant science results, from outbursting young stars to subluminous Type IIP supernovae to candidate tidal disruption events. We discuss future directions of this approach, including the possible roles of crowdsourcing and the scalability of machine learning to future surveys such a the Large Synoptical Survey Telescope (LSST).
1. Introduction
Synoptic surveys generate data faster than humans can inspect, making computational discovery and classification necessary for real-time time-domain astrophysics. The paper presents a machine-learned framework designed to automate these tasks while retaining calibrated, reproducible classifications.
- Motivation: Synoptic imaging data volumes now exceed the capacity of astronomers and citizen scientists to inspect in real time.This shift increases the role of software and hardware frameworks in the real-time loop.
- Discovery challenge: Image artifacts and correlated candidate parameters make autonomous discovery of real transients and variable stars difficult.Simple thresholding can be suboptimal, while matched filters recover only classes with suitable expected light-curve behavior.
- Motivation: Real-time classification keeps streaming surveys informed of progress while time-variable sources are still accumulating data.This can support changes in survey direction when scientifically warranted.
- Motivation: Computational discovery and classification can operate faster and more reproducibly than human-centered workflows at high data rates.The framework also supports rerunning new classification schemes and calibrating classification probabilities against survey ground truth.
- Contribution: The paper develops a machine-learned framework for discovering transients and variable stars and producing autonomous initial classifications from expert-trained input.It also describes calibration of false-negative and false-positive rates using survey data and applies the approach to PTF observations.
2. Discovery on Images
PTF discovers changing sources through image differencing, then uses machine learning to distinguish astrophysical candidates from subtraction artifacts. The framework calibrates discovery performance with real/bogus scores, multi-epoch requirements, and validation against known sources.
- Image-difference discovery: Image differencing subtracts a deep reference image from each aligned, flux-scaled exposure, then extracts sources from the resulting difference image.This approach is favored for faint variables and events near or within galaxies, despite its computational cost.
- Image-difference discovery: Only about 1 in 1000 extracted ≥5-σ PTF difference-image candidates is astrophysically real, because artifacts arise from reduction, alignment, scaling, detector, and cosmic-ray effects.This extreme imbalance makes simple candidate presentation and thresholding inadequate.
- Real or bogus classification: Machine learning replaces real-time human decisions with statistical real/bogus statements that can be combined across multiple epochs to identify discoveries.Candidate labels may come from human scans, artificial sources, or ground-truth catalogs, each with distinct advantages and limitations.
- Validation: Asteroid-associated candidates validate the classifier but may include faint bad-subtraction contamination, and brighter candidates generally receive higher realbogus values.The asteroid sample provides a practical real-source check, though its sky distribution may not represent sources near stars and galaxies.
- Multi-epoch discovery: With two-candidate discovery and FNR = 0.4, FPR = 0.01, the expected purity is 78% and efficiency is 36% for finding real sources.The estimates ignore correlations among realbogus values, while PTF observes correlated bogus candidates near problematic sky locations.
3. Classification
Oarical separates source discovery from classification, using contextual and time-domain information to assign increasingly specific labels with limited observations. On PTF data, machine-learned classification achieved strong performance overall and improved substantially when SDSS context was available, although training-sample biases constrain interpretation.
- 3. Classification: Oarical classifies newly discovered sources using contextual information and time-domain features designed to work with only a few imaging epochs.The framework emphasizes rapid preliminary classification, particularly for transient and explosive events.
- 3. Classification: The PTF Type classifier obtained a 3.8% overall error rate, decreasing to 1.7% with SDSS coverage and increasing to 9.4% without it.These error rates were measured using 10-fold cross validation on 1573 Transient and 380 VarStar sources.
- 3. Classification: At 90% purity, random forests reached 96.6% efficiency for variable-star classification and 99.7% for transient classification.For SDSS sources, the corresponding efficiencies were 96.6% and 100%.
- 3. Classification: The five-class robotclass model achieved a 6.5% error rate, with 3.7% error for SDSS sources versus 14.1% without SDSS coverage.Recovery was 97% for AGN-cnSN-TDE, 93% for SN/Nova, and 89% for VarStar-Periodic; class imbalance harmed smaller categories.
- 3. Classification: Additional PTF observations mainly improved classification when SDSS features were unavailable, reducing the no-SDSS error rate to 13.2% after 30 days and 12.8% after 90 days.Performance for sources in SDSS did not change significantly with additional PTF observations.
- 3. Classification: Training-sample biases limit interpretation because catalog labels favor brighter sources and can overlap with SDSS information used by the classifier.The reported results should conservatively be understood as classification of sources already observed to vary significantly and established as astrophysical variables or transients.
4. Discussion and Conclusions
The framework demonstrated end-to-end, near-real-time discovery and classification on PTF data without humans in the real-time loop, while exposing important boundaries for future surveys and source classes.
- Discussion and Conclusions: The framework integrates autonomous acquisition, near-real-time image processing, database markup, and robotic follow-up without humans in the real-time loop.PTF images are acquired autonomously, transported and processed near real time, and used to trigger follow-up observations.
- Discussion and Conclusions: A Keck spectrum was obtained 29 minutes after Oarical discovery, and analysis of the peculiar Type Ia supernova was published less than 18 hours after PTF observation.
- Discussion and Conclusions: 529 spectroscopically confirmed SNe discovered autonomously by Oarical since April 2010 represented more than half of the PTF collaboration's lifetime supernova discoveries.
- Discussion and Conclusions: Future improvements include larger ground-truth samples, image-difference metrics, improved astrometric ties, and automated forced-aperture and PSF photometry.
- Discussion and Conclusions: Machine-learned transient-versus-variable-star classifications were reasonably predictive, with errors at the 5% level.
- Discussion and Conclusions: Open questions concern anomalous sources, citizen-scientist roles, transfer across surveys, feedback from follow-up, and controlling bias during relearning.
- Discussion and Conclusions: Automatic discovery workflows are expected to require survey-specific tuning, whereas classification as a service may develop as a more general third-party framework.