Source-linked AI summary
Identifying Exoplanets with Deep Learning: A Five Planet Resonant Chain around Kepler-80 and an Eighth Planet around Kepler-90
Christopher J. Shallue, Andrew Vanderburg
TL;DR
Earth-sized planets can produce signals near Kepler’s detection limit, motivating automated candidate assessment. This paper uses deep-learning models to distinguish planets from false positives and applies them to new transit candidates, achieving strong ranking performance and identifying highly confident candidates for validation.
Problem
Earth-like planets may have shallow, infrequent transits near Kepler’s detection limit, complicating reliable candidate classification.
Method
The paper uses deep-learning neural networks with fully connected architectures and model averaging to classify Kepler transit candidates.
Results
98.8% of the time, the model ranked true planet candidates above false positives on the test set; four new TCEs received planet probabilities of at least 0.9.
Takeaways & Limitations
Applying the model to known Kepler multi-planet systems produced highly ranked candidates, including a new 14.64-day TCE exterior to the five known planets around KIC 4852528.
Takeaways & Limitations
The model still requires human supervision for reliable classifications and is not ready to blindly classify TCEs for planet-occurrence estimates.
Abstract
from arXiv · showhide
NASA's Kepler Space Telescope was designed to determine the frequency of Earth-sized planets orbiting Sun-like stars, but these planets are on the very edge of the mission's detection sensitivity. Accurately determining the occurrence rate of these planets will require automatically and accurately assessing the likelihood that individual candidates are indeed planets, even at low signal-to-noise ratios. We present a method for classifying potential planet signals using deep learning, a class of machine learning algorithms that have recently become state-of-the-art in a wide variety of tasks. We train a deep convolutional neural network to predict whether a given signal is a transiting exoplanet or a false positive caused by astrophysical or instrumental phenomena. Our model is highly effective at ranking individual candidates by the likelihood that they are indeed planets: 98.8% of the time it ranks plausible planet signals higher than false positive signals in our test set. We apply our model to a new set of candidate signals that we identified in a search of known Kepler multi-planet systems. We statistically validate two new planets that are identified with high confidence by our model. One of these planets is part of a five-planet resonant chain around Kepler-80, with an orbital period closely matching the prediction by three-body Laplace relations. The other planet orbits Kepler-90, a star which was previously known to host seven transiting planets. Our discovery of an eighth planet brings Kepler-90 into a tie with our Sun as the star known to host the most planets.
2.1. Introduction
Deep learning learns increasingly complex classification features automatically, while neural-network architectures transform inputs through connected, convolutional, pooling, and nonlinear layers.
- Deep learning builds increasingly complex features through computational layers and often outperforms models based on hand-designed features.
- A fully connected network passes neuron activations hierarchically from inputs through hidden layers to output predictions.
- 2.2. Fully Connected Neural Networks: Hidden-layer activations combine learned weights and biases before applying a nonlinear activation function, while binary outputs use sigmoid probabilities.
- Convolutional networks use local feature maps and pooling to exploit spatial structure and reduce sensitivity to small input translations.
- 2.3. Convolutional Neural Networks: Convolutional layers apply learned kernels and activation functions to input vectors, producing feature maps that detect local features.
- 2.3. Convolutional Neural Networks: Pooling aggregates neighborhood values, typically by mean or maximum, across regions separated by a stride length.
2.4. Neural Network Training
The model is trained by minimizing binary cross-entropy against manually assigned TCE labels, with separate validation and test partitions used for development and evaluation.
- 2.4. Neural Network Training: Binary cross-entropy measures disagreement between model predictions and true training labels and is used as the classification cost function.
- 2.4. Neural Network Training: Gradient-based updates iteratively adjust initially random model parameters to reduce the cost function.
- 3. Creating Our Training Set: The training set contains 3,600 planet candidates, 9,596 astrophysical false positives, and 2,541 non-transiting phenomena.
- 3. Creating Our Training Set: Labels are binarized as planet versus not planet, combining astrophysical false positives and non-transiting phenomena in the latter class.
- 3. Creating Our Training Set: The authors assume catalog labels are ground truth despite finding several mislabeled examples through manual examination.
- 2.4. Neural Network Training: The data are randomly split into 80% training, 10% validation, and 10% test subsets.
3.2. Light Curves
Kepler light curves are folded, flattened, binned, normalized, and represented through global and local views before being classified by alternative neural-network architectures.
- 3.2. Light Curves: Kepler light curves contain flux measurements at 29.4-minute intervals for up to four years, or approximately 70,000 points.
- 3.2. Light Curves: Inputs are generated by folding each flattened light curve on the TCE period and binning it into a one-dimensional vector.
- 3.2. Light Curves: Overlapping bins can reduce scatter and make some transits more visible by using intervals wider than the distance between their midpoints.
- 3.3. Input Representations: The global view spans the entire light curve at fixed length, whereas the local view covers a fixed-duration window around the detected transit.
- 3.3. Input Representations: The global representation treats short- and long-period TCEs equally but may compress long-period transits into very few bins and omit secondary-eclipse information.
- 3.2. Light Curves: All light curves are normalized to median 0 and minimum value -1, giving TCEs a fixed transit depth.
- 4.1. Neural Network Architectures: The models compare global-only, local-only, and combined views across linear, fully connected, and convolutional architectures.
- 4.1. Neural Network Architectures: Combined-view fully connected and convolutional models process global and local vectors through separate branches before shared layers combine them.
4.2. Training Procedure
The models were trained and evaluated using standard classification metrics, threshold-based decisions, and multiple input representations. The convolutional architecture using both global and local views performed best on the held-out test set.
- Training procedure: Hyperparameters were tuned with black-box optimization to maximize validation-set AUC across input representations, architectures, and training settings.Each study trained several thousand models.
- Training procedure: Model averaging used 10 independently initialized copies, whose outputs were averaged for predictions.The ensemble reduces variance between individual models and can improve performance when training data are limited.
- Evaluation metrics: AUC measures the probability that a randomly selected planet is ranked above a randomly selected false positive, independently of classification threshold.Precision, recall, and accuracy depend on the selected probability threshold.
- Test-set performance: The convolutional model achieved recall of 0.95 at precision of 0.90 with an appropriate classification threshold.This corresponds to identifying 95% of real planets while 90% of planet classifications were true planets.
- Test-set performance: The convolutional architecture performed best on the test set when using both global and local input views.The test set contained 1,523 TCEs not used for training or hyperparameter decisions.
- Input representations: Using both global and local views improved performance over either view alone for every architecture.Linear and fully connected models favored local over global views individually, whereas the convolutional model showed the opposite pattern.
5.2. Best Model Configuration
The best model combines global and local light-curve views in a convolutional architecture selected using validation-set AUC. Visualizations indicate that it focuses on transits and secondary eclipses while organizing candidates in a learned representation space.
- Model configuration: The best model uses a convolutional architecture with both global and local input views.Its hyperparameters were selected to maximize validation-set AUC.
- Input views: The global view contains 2,001 points, while the local view contains 201 points spanning four transit durations on either side of the event.The views are parameterized using the TCE period and duration.
- Prediction heatmaps: Occlusion heat maps show that the transit is the most important input region for a planet candidate.The model’s prediction drops to zero when the transit is occluded but remains largely unchanged when other regions are blocked.
- Prediction heatmaps: Occluding a secondary eclipse increases the planet prediction because it hides evidence that the signal is an eclipsing binary.Predictions remain low, at approximately 0.4 or below, even after the secondary eclipse is blocked.
- Architecture notation: Figure 7 denotes convolutional layers as conv<kernel size>-<number of feature maps>, pooling layers as maxpool<window length>-<stride length>, and dense layers as FC-<number of units>.These labels describe the layer types and their principal dimensions.
- Learned representation: The final hidden layer represents light curves in a space where a linear boundary can separate planet candidates from false positives.t-SNE projects these high-dimensional activations into two dimensions for visualization.
5.4. Performance on Simulated Data
On simulated Kepler data, the model was permissive for injected planets but less effective than the Robovetter at rejecting several false-positive classes, especially at low MES.
- Injected Group 1: 36,826 of 45,377 recovered simulated planets were classified as planets by the model, versus 38,668 by the Robovetter.
- Injected Group 1: At MES greater than 10, the model typically assigned high confidence to injected planets, but confidence declined steeply from MES=10 to MES=5.MES=5–10 is the likely regime for Earth-sized planets in temperate orbits and lies near the Kepler pipeline’s detection limit.
- Injected Group 2: The model often classified off-target injected signals as planets because it lacked pixel-location information such as centroid shifts.
- Injected Group 3: About 65% of simulated eclipsing-binary TCEs were classified as planets, compared with about 37% by the Robovetter.The model rejected most systems when the secondary signal was stronger than the primary, but passed most when the secondary was weaker.
- Inverted Group: 3.2% of inverted-group false positives were classified as planets, compared with about 0.4% by the Robovetter.The flattening procedure occasionally made non-transit-like variability resemble transits.
- Summary: The model did not yet match the Robovetter on simulated false positives and would require human supervision before blind occurrence-rate classification.
5.5. Comparisons to Other Automatic Vetting Systems
The model was compared with the Robovetter and other automated systems using different input strategies and evaluation assumptions. It achieved higher ranking AUC than the Robovetter under the paper’s comparison setup, while the Robovetter had higher accuracy.
- Robovetter comparison: The model achieved AUC 0.988 versus 0.974 for the Robovetter, while accuracy was 0.960 versus 0.974.The comparison assumed the training labels were ground truth and treated TCEs absent from the DR25 KOI table as rejected.
- Related systems: The Autovetter classifies TCEs using pipeline-derived statistics, whereas this model directly represents light-curve shapes.The two approaches are not mutually exclusive, and the authors planned to explore auxiliary input features.
- Related systems: The Autovetter could not rank the paper’s new TCEs because those candidates came from an alternative detection pipeline.
- Related systems: Armstrong et al. reported accuracy of 0.863 on planet candidates and 0.875 on false positives, versus 0.949 and 0.963 for this model.
6.1. Transit Search
The authors searched known Kepler multi-planet systems for additional transit signals after removing known planets, using a deliberately permissive detection threshold and automated post-processing.
- Search design: The search covered 670 stars across systems containing two to seven known planet candidates.
- Search design: Known-planet transits were removed before searching for new candidates with a BLS-based pipeline.The pipeline flattened light curves, searched a BLS periodogram, and recorded TCEs above the selected signal-to-noise threshold.
- Detection threshold: The search used S/NBLS = 5, below the typical K2 threshold of 9 and the Kepler pipeline threshold of MES >7.1.The lower cutoff increased the number of spurious TCEs.
- False-positive filtering: Post-processing removed detections with transit durations shorter than 29.4 minutes or fewer than three complete transits.These filters targeted implausibly short events and alignments involving gaps, instrumental noise, or cosmic rays.
- Scope: The pipeline was optimized for shorter periods of roughly 40 days or less, so most new detections were short-period candidates.
- Search yield: The search produced 513 new TCEs, nearly one per searched star, making manual vetting impractical at full Kepler scale.The neural network was therefore used to rank the TCEs by planet probability.
6.2. Summary of New Highly-Ranked TCEs
The model ranked new TCEs from the multi-planet search, identifying a small set of high-confidence candidates for additional validation. These candidates included signals associated with possible period aliases, additional planets, resonant systems, and unresolved light-curve scatter.
- Individual candidates: KIC 11442793, already hosting seven transiting planets, yielded an additional 14.4-day TCE with S/N 8.7.
- Individual candidates: Additional candidates included TCEs at 10.9 days, 27.1 days, and 11.1 days in KIC 8480285, KIC 11568987, and KIC 10337517, respectively.The KIC 11568987 signal had S/N = 9.8 and a depth corresponding to a roughly Earth-sized planet.
- Individual candidates: KIC 4852528 yielded an 8.6-S/N TCE at 14.64 days exterior to five known planets in a system with three-body resonances.
- Individual candidates: The KIC 5972334 candidate had S/N = 10.2 and a 6.02-day period but showed extra in-transit scatter requiring further understanding.
- Individual candidates: KIC 11030475 yielded a 4.75-day TCE matching the transit timing, depth, duration, and half-period of KOI 2248.02, suggesting the detected period may be the candidate’s true period.
- Individual candidates: KIC 4548011 yielded a 13.95-day TCE with S/N = 9.7 that would likely be sub-Earth-sized if real and was later listed as KOI 4288.04.
6.3. Vetting New TCEs
The authors vet new threshold-crossing events with centroid, photometric, and contamination checks to distinguish astrophysical transits from false positives. Kepler-80 and Kepler-90 pass these tests, while Kepler-647 remains uncertain because its transit depth changes over the mission.
- Image Centroid Analysis: No significant centroid motion was detected during transit for the three targets inspected.For Kepler-80, measured shifts were 0.1 and 0.05 milliarcseconds, each with 0.13-milliarcsecond uncertainties.
- Image Centroid Analysis: The authors conclude that none of the visually identified companions to the three hosts produces the new transit signals.Centroid constraints place the signals co-spatial with the target stars rather than nearby companions.
- Photometric Tests: Secondary-eclipse, aperture, even-odd, quarterly, ephemeris, and SAP/PDCMAP checks found no significant contaminating or instrumental signatures.These tests target eclipsing binaries, scattered background light, CCD-position effects, matching signals, and residual systematics.
- Photometric Tests: Kepler-80 and Kepler-90 signals appear astrophysical and are treated as planet candidates after the vetting checks.Kepler-647 was not validated with high confidence because its transit depth appeared to increase over the mission, possibly indicating column anomaly.
6.4. False Positive Probability Calculations
The authors estimate false-positive probabilities for the two vetted candidates with vespa, incorporating imaging, eclipse, and centroid constraints. Multiplicity information then reduces both probabilities below the validation threshold.
- False Positive Probability Calculations: The vespa analysis incorporates high-resolution imaging, secondary-eclipse limits, and centroid constraints on the transit sources.These inputs constrain astrophysical false-positive scenarios that can resemble the observed signals.
- False Positive Probability Calculations: Vespa estimates false-positive probabilities of about 0.002 for Kepler-80 and 0.005 for Kepler-90.Both values are below the 1% threshold often used to validate Kepler planets.
- False Positive Probability Calculations: Accounting for the candidates’ multi-planet systems lowers both false-positive probabilities to less than 10^-4.The authors apply a multiplicity boost because multi-planet-system candidates are less likely to be false positives.
- False Positive Probability Calculations: The authors therefore validate the candidates as Kepler-80 g and Kepler-90 i.The validation follows the vespa probabilities and the multiplicity adjustment.
6.5. Transit Fits
After validation, the authors fit the complete Kepler long-cadence light curves with transit models and use affine-invariant MCMC to estimate orbital and planetary parameters with uncertainties.
- Transit Fits: The authors fit Mandel & Agol transit models to the full Kepler long-cadence light curves.The fitted quantities include orbital periods, transit times, inclinations, scaled semimajor axes, and planetary parameters.
- Transit Fits: Affine-invariant ensemble MCMC explores parameter space and estimates uncertainties for Kepler-80 g and Kepler-90 i.Each fit used 100 walkers evolved for 20,000 links, with the first 10,000 discarded as burn-in.
- Transit Fits: All fitted parameters achieved Gelman-Rubin potential reduction factors below 1.2.The authors use this threshold as their convergence check for the MCMC chains.
7.1. Newly Validated Planets
The paper identifies two validated planets in high-multiplicity systems: Kepler-80 g extends a resonant chain, while Kepler-90 i becomes the system’s eighth known planet. Their orbital configurations provide additional tests and context for planetary-system architecture.
- Newly Validated Planets: Two new exoplanets were identified using a transit search combined with the machine-learning vetting pipeline.Both planets belong to high-multiplicity systems: Kepler-80 has five other planets and Kepler-90 has seven.
- Kepler-80 g: Kepler-80 g is the outermost planet and likely completes a five-planet resonant chain.Adjacent period ratios are near 2:3 or 3:4, and consecutive trios satisfy Laplace relationships with a shared super-period near 192 days.
- Kepler-80 g: Kepler-80 g’s orbital period closely matches the predicted period for an additional outer planet in three-body resonance with Kepler-80 b and c.The authors state that further dynamical analysis is required to confirm the resonant state.
- Kepler-90 i: Kepler-90 i has a 14.4-day period and a radius of 1.3 R⊕, placing it near the system’s two shortest-period, similarly small planets.Kepler-90 b, c, and i may be rocky based on their radii, while Kepler-90 i’s transit duration is shorter than expected from the system’s period-duration trend.
- Kepler-90 i: Kepler-90 becomes the first known exoplanetary system with eight planets and ties the Sun for the most planets known around a star.The authors discuss its compact, orderly architecture as a window into planetary formation and migration.
7.2. Looking Ahead
The authors identify several ways to improve their deep-learning vetting model, particularly for low-signal-to-noise candidates and known false-positive classes. These improvements support the longer-term goal of more accurate planetary occurrence-rate estimates while acknowledging current performance limits.
- Motivation: Lower signal-to-noise thresholds recover more planets but produce too many false-positive TCEs for practical manual examination.Automatic vetting is therefore required to study Kepler’s low-signal-to-noise regime thoroughly.
- Future improvements: The current training set contains about 15,000 labeled TCEs, with only 8% having MES less than 10, and performance drops substantially for real candidates at MES ≲10.Adding simulated or unlabeled data could enlarge training and likely improve performance on low-MES candidates and several false-positive classes.
- Future improvements: Improving light-curve flattening could reduce bogus transits caused by high-frequency stellar variability.The current flattening method occasionally creates signals that are classified as likely planets.
- Future improvements: Potential input improvements include centroid information, segmented local views, secondary and tertiary views, auxiliary stellar features, and robust means.Robust means could theoretically boost the model’s input S/N by up to 25%, while auxiliary TCE features showed no validation improvement.
- Conclusion and outlook: The model ranks true planet candidates above false positives 98.8% of the time, but remains less effective than Robovetter at rejecting some simulated false positives, including weak secondary eclipses.The authors aim to address these failure modes before using the technique to improve Earth-like planet occurrence-rate estimates.