Source-linked AI summary
Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding
Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, Tom Soderstrom
TL;DR
Spacecraft anomaly monitoring must handle complex, high-volume telemetry while balancing false positives and false negatives. This paper evaluates LSTM-based telemetry prediction with unsupervised nonparametric dynamic thresholding, finding that pruning substantially improves precision and that the approach can identify contextual anomalies.
Problem
Spacecraft operations require anomaly detection across many telemetry channels, while existing monitoring approaches provide limited coverage and must balance false positives against false negatives.
Method
The paper combines LSTM-based one-step telemetry prediction with unsupervised nonparametric dynamic thresholds and pruning to detect anomalous smoothed prediction errors.
Results
Pruning increased overall precision from 48.9% to 87.5% while reducing recall from 84.8% to 80.0%, and LSTM models achieved 5.9% average normalized absolute error for one-step-ahead prediction.
Takeaways & Limitations
The results demonstrate the viability of LSTMs and dynamic thresholding for spacecraft telemetry anomaly detection, including detection of a majority of contextual anomalies.
Takeaways & Limitations
The evaluation assumes one anomaly every five days and omits streams without anomalies, making implemented-system precision lower than the experimental precision; command representations may also be insufficient for missions without routine operations.
Abstract
from arXiv · showhide
As spacecraft send back increasing amounts of telemetry data, improved anomaly detection systems are needed to lessen the monitoring burden placed on operations engineers and reduce operational risk. Current spacecraft monitoring systems only target a subset of anomaly types and often require costly expert knowledge to develop and maintain due to challenges involving scale and complexity. We demonstrate the effectiveness of Long Short-Term Memory (LSTMs) networks, a type of Recurrent Neural Network (RNN), in overcoming these issues using expert-labeled telemetry anomaly data from the Soil Moisture Active Passive (SMAP) satellite and the Mars Science Laboratory (MSL) rover, Curiosity. We also propose a complementary unsupervised and nonparametric anomaly thresholding approach developed during a pilot implementation of an anomaly detection system for SMAP, and offer false positive mitigation strategies along with other key improvements and lessons learned during development.
1 INTRODUCTION
Spacecraft anomaly detection is essential but increasingly difficult as telemetry volume, mission complexity, and data heterogeneity grow. The paper combines interpretable LSTM prediction with dynamic nonparametric thresholding and evaluates the approach on expert-labeled SMAP and MSL data.
- Spacecraft operations monitor thousands of telemetry channels because missed hazards can cause partial or complete spacecraft loss.
- Existing alarms, manual analyses, expert systems, and nearest-neighbor methods require substantial expert knowledge and human effort to maintain.
- Increasing telemetry volumes and more complex, time-constrained missions intensify the need for scalable anomaly detection.NISAR is cited as generating around 85 terabytes of data per day.
- Spacecraft telemetry anomaly detection must address scarce labels, non-stationarity, contextual dependence, heterogeneity, noise, dimensionality, and interpretability.The system must also balance false positives and false negatives for the operational scenario.
- The paper uses LSTM recurrent neural networks for prediction and proposes unsupervised, nonparametric dynamic thresholding to evaluate residuals.The thresholding approach is designed for data streams with varying behaviors and value ranges.
- Experiments use expert-labeled Incident Surprise, Anomaly reports from the SMAP satellite and MSL rover, Curiosity.
2 BACKGROUND AND RELATED WORK
Prior spacecraft anomaly detection relies heavily on simple limits or specialized methods that face precision, scalability, resource, and generalizability constraints. LSTMs offer a temporal modeling alternative for telemetry streams and have already been applied to prediction-based anomaly detection.
- Time-series anomalies are categorized as point, collective, or contextual according to whether individual values, sequences, or local context are anomalous.
- Out-of-limits, clustering, expert-system, nearest-neighbor, and neural-network approaches have all been applied to anomaly detection.
- Out-of-limits methods remain widely used because they are computationally inexpensive, broadly applicable, and easy to understand.
- Prior systems can achieve high recall while producing many false positives, limiting operational usefulness.The STS-115 example detected all engineer-identified anomalies but also flagged many non-anomalous events.
- Earlier aerospace methods often used limited telemetry representations, constrained sensor coverage, or no information beyond prior telemetry values.ELMER, for example, was used only for 10 temperature sensors because of onboard resource limitations.
- Existing approaches have not demonstrated results and generalizability compelling enough to justify widespread adoption across missions.
- LSTMs model relationships between past and current values, learn nominal system behavior, and support prediction-based anomaly detection.LSTM predictions are compared with actual stream values using detection rules.
3 METHODS
The approach uses per-channel LSTMs to predict telemetry from historical values and command information, then applies dynamic unsupervised thresholds, anomaly scoring, pruning, and false-positive mitigation.
- Overview: The system uses LSTMs to predict high-volume telemetry from normal command and telemetry sequences, then evaluates prediction errors for anomalies.The approach is unsupervised and designed to assess hundreds to thousands of diverse telemetry streams.
- Single-Channel Models: A separate model is created for each telemetry channel, preserving channel-level traceability and enabling anomalies to be aggregated into subsystem groupings.Channel-level modeling also supports granular control over training and overfitting issues.
- Predicting Values for a Channel: For each channel, the model uses a sequence of prior observations and predicts one step ahead, with encoded command information included in the inputs.The prediction length is set to lp = 1, producing a single scalar prediction for each step.
- Errors and Smoothing: Prediction errors are computed from absolute differences between true and predicted telemetry values, collected over history, and smoothed with an exponentially weighted moving average.Smoothing dampens sharp error spikes caused by abrupt but normal changes that LSTM predictions may not capture perfectly.
- Threshold Calculation and Anomaly Scoring: The unsupervised threshold selects the candidate that most reduces the mean and standard deviation of smoothed errors without statistical assumptions, then assigns severity scores to anomalous sequences.Candidate thresholds are based on z values representing positive standard deviations above the smoothed-error mean; values from two to ten worked well experimentally.
- Pruning Anomalies: Anomaly pruning reduces false positives and resource costs by comparing descending sequence maxima and retaining anomalies only when successive percent decreases exceed a minimum threshold.User labels can also calibrate the minimum anomaly score near the upper bound of confirmed false positives, improving precision for infrequent but regular spacecraft behaviors.
4 EXPERIMENTS
The experiments evaluate LSTM-based telemetry prediction and nonparametric thresholding on expert-labeled SMAP and MSL anomaly data. Results show that pruning improves precision substantially, while performance varies by anomaly type and spacecraft.
- Experimental Data: Expert-labeled telemetry anomalies from SMAP and MSL were mined from ISA reports and manually labeled for specific anomalous channel ranges.Closely resembling anomalous sequences and channels were reduced to create a diverse, balanced set.
- Anomaly Types: The evaluation distinguishes point anomalies from contextual anomalies, with contextual anomalies requiring temporal information beyond limits or distance-based methods.Contextual and collective anomalies were combined into the contextual category for analysis.
- Evaluation: Anomaly decisions were scored by overlap with labeled sequences: overlapping predictions counted as true positives, uncovered labels as false negatives, and non-overlapping predictions as false positives.Scoring did not adjust for detection earliness or distance from labeled regions.
- Model Parameters: The thresholding experiments varied z from 2.5 to 10.0 with p = 0.13; reasonable results occurred for 0.05 < p < 0.20.The p parameter controls the precision–recall balance when separating subsequent anomalies.
- Results: LSTM predictions combined with nonparametric thresholding and pruning achieved the best F0.5 results, with 5.9% average normalized absolute one-step prediction error.The result identifies the strongest evaluated combination while separately reporting prediction accuracy.
- Results: Pruning reduced recall from 84.8% to 80.0% but increased precision from 48.9% to 87.5%, while Gaussian thresholding performed worse under violated normality assumptions.Thresholding cannot compensate for poor predictions whose smoothed errors lack an anomaly signal.
5 DEPLOYMENT
The deployed system pilots automated telemetry monitoring for SMAP while identifying prediction, false-positive, interface, and cross-channel dependency challenges that constrain broader adoption.
- Deployment: The system monitors over 700 SMAP channels in near real time, with models trained offline every three days, and has identified several confirmed anomalies.High precision remains essential because more than one million telemetry values are processed daily and false positives consume operations engineers’ attention.
- Future Work: Improved feature engineering is planned to strengthen telemetry predictions and establish feasibility for large-scale automated monitoring.The authors describe pilot deployment and experimental results as key milestones toward this goal.
- Future Work: Current command encoding omits command nature, so granular command and event information may be necessary for accurate prediction on missions without routine operations.Such missions would also require automatically selecting training data from periods with similar planned activities.
- Future Work: Future work includes refining false-positive mitigation and improving interfaces for anomaly review, investigation, and expert labeling.These improvements target practical use of the system by operations personnel.
- Future Work: Telemetry-channel interactions and dependencies remain insufficiently addressed, limiting automated insight into complex system behaviors and anomalies.A visual interface partially addresses this issue, but the authors call for a more mathematical and automated treatment of channel correlations.
6 CONCLUSION
The paper demonstrates LSTM-based spacecraft telemetry prediction and proposes dynamic thresholding to address anomaly-detection challenges involving scale, interpretability, precision, and complexity. It also identifies improvement areas and releases expert-labeled data and open-source implementations.
- 6 CONCLUSION: The paper demonstrates that LSTMs can predict spacecraft telemetry while addressing interpretability, scale, precision, and complexity challenges.The conclusion frames the approach as relevant to anomaly detection scenarios beyond spacecraft operations.
- 6 CONCLUSION: The proposed dynamic thresholding approach avoids reliance on scarce labels and false parametric assumptions.The approach is presented as a novel component of the paper’s anomaly-detection methodology.
- 6 CONCLUSION: The authors identify key areas for improvement and further evaluation as they expand capabilities across spacecraft.The conclusion does not claim that all deployment challenges have been resolved.
- 6 CONCLUSION: The paper makes public a large real-world, expert-labeled spacecraft anomaly dataset and open-source implementations of its methodologies.These resources accompany the paper’s methodological contributions.