PaperlayerView exact revision on arXiv

Source-linked AI summary

Can Large Language Models Explain Flight Safety Events? A Prior-Guided Semantic LLM-based Approach

Lu Xu, Xu Li, Linjiang Zheng, Fan Li, Riquan Zhang, Jiaxing Shang

arXiv:2608.18017v1cs.AI

TL;DR

Flight safety analysis needs explanations of underlying pilot-control causes, not only event detection. FlightLLM combines semantic flight-data representations, statistical prior guidance, and structured prompting, achieving competitive classification while producing direct explanations for hard-landing causes.

  • Problem

    Flight safety analysis still needs interpretable explanations of complex event causes and pilot control behavior beyond identifying what happened.

  • Method

    FlightLLM semantically discretizes engineered flight features, injects CatBoost predictions as prior guidance, and uses contrastive few-shot learning with aviation-specific prompts.

  • Results

    On 704 real-world A320 flights, FlightLLM achieves competitive classification performance and generates direct, evidence-based explanations for hard-landing causes.

  • Takeaways & Limitations

    FlightLLM provides a reference for interpreting hard-landing causes in pilot training and flight-operation quality assurance.

  • Takeaways & Limitations

    The study uses pre-trained models without hard-landing-specific fine-tuning, and extending analysis across events requires manually modified prompts.

Abstract

from arXiv · show

Improving flight safety with flight data requires not only accurate detection of risk events, but more importantly, clear interpretation of their underlying causes at the level of pilot control behavior. Existing explainable AI techniques, such as feature importance maps, often require considerable domain knowledge to translate them into operationally meaningful explanations. Large Language Models (LLMs), which excel at language reasoning, bring a promising solution to this issue. However, applying LLMs in this domain presents key challenges such as modal inconsistency, limited classification ability, scarcity of task-specific data for fine-tuning, and lack of domain knowledge. To overcome these challenges, we propose FlightLLM, a prior-guided semantic LLM-based approach for interpretable flight safety analysis. Specifically, we first perform feature engineering to address modal inconsistency, combining statistical descriptors with physically meaningful flight indicators. This representation is further processed by a Semantic Discretization module, which converts abstract numerical patterns into qualitative descriptions that are more compatible with language reasoning. In addition, since LLMs are not inherently strong classifiers, CatBoost is incorporated as a statistical expert, and its prediction results are injected into the prompt as prior guidance. A contrastive few-shot learning strategy is further adopted to compensate for limited data. Finally, we design structured prompts to embed aviation-specific knowledge into the inference process. Using hard landing, a representative risk event with complex causal mechanisms, as an anchor point, we evaluate FlightLLM on a dataset of 704 real-world A320 flight samples. Experimental results show that the proposed approach achieves competitive classification performance while generating direct and reasonable explanations for event causes.

I. INTRODUCTION

The introduction motivates interpretable flight-safety analysis beyond event detection and presents FlightLLM, which combines semantized QAR features, statistical prior guidance, contrastive few-shot learning, and structured prompts. Using hard landing as an anchor, the approach targets competitive classification and direct, evidence-based causal explanations on 704 real-world A320 flights.

  • Motivation: Flight safety analysis must move beyond identifying events toward uncovering complex causes that can support risk reduction and aviation-safety improvement.The paper frames causal analysis as important for reducing incident rates and enhancing aviation safety.
  • Challenges: LLMs can generate textual explanations but face cross-modal inconsistency, high-dimensional QAR inputs, limited classification ability, scarce task-specific data, and insufficient aviation knowledge.These challenges motivate feature engineering, semantic discretization, statistical classification guidance, few-shot learning, and structured prompts.
  • Proposed approach: FlightLLM uses hard landing as an anchor event and combines semantic feature processing, statistical-expert guidance, contrastive few-shot learning, and structured aviation prompts.The framework is organized around four key challenges and injects domain knowledge to mitigate hallucination.
  • Contributions: FlightLLM links flight-safety classification with causal diagnosis by converting abstract risk assessment into practical actions for pilot training and flight-operation quality assurance.The stated contribution emphasizes intelligible explanations of hard-landing causes rather than classification alone.
  • Experimental evidence: 704 real-world A320 flights support the approach’s reported high accuracy and precision alongside direct, evidence-based explanations of hard-landing causes.The explanations are presented as references for lowering risks in future flights.

II. RELATED WORK … III. METHODOLOGY

Related work has developed multiple model-based interpretability techniques for flight safety and begun connecting LLMs with time-series analysis, but explanations still require domain experts to form causal narratives. FlightLLM addresses this gap through a five-module pipeline combining aviation data processing, semantic representation, statistical expertise, retrieval, and LLM prompting.

  • A. Data-Driven Flight Safety Analysis and Interpretability: Landing-phase safety analysis is a central aviation-safety focus because landing accounts for 37% of fatal accidents despite occupying only 1% of flight time.The passage links this disproportionate risk to the importance of event attribution for mitigating landing-related hazards.
  • A. Data-Driven Flight Safety Analysis and Interpretability: LIME and SHAP have been used with models such as TabNet and XGBoost to identify critical factors in wind-shear and runway-safety events.The cited studies combine prediction frameworks with post-hoc feature-attribution methods.
  • A. Data-Driven Flight Safety Analysis and Interpretability: CAM-based approaches such as IMTCN explain hard-landing predictions by highlighting discriminative temporal regions in QAR time-series classifications.IMTCN integrates multiple Temporal Convolutional Networks with an improved CAM mechanism.
  • A. Data-Driven Flight Safety Analysis and Interpretability: Attention-based and hybrid architectures provide fine-grained interpretability, but experts must translate highlighted regions or attention distributions into causal narratives.Examples include dual-stage attention, RNN-Transformer, and GCN-based approaches for landing-safety analysis.
  • B. Large Language Models for Time Series: LLM research has extended language-model reasoning toward time-series tasks, including financial forecasting and strategies that adapt LLM architectures or transform temporal inputs.The cited review identifies two primary strategies for using LLMs in time-series analysis.
  • III. METHODOLOGY: FlightLLM comprises five modules: Data Preprocessing and Feature Engineering, Semantic Discretization, Statistical Expert Hinting, Dynamic Context Retrieval, and Prompt Construction and LLM Invocation.Modules (i)–(iv) form the core pipeline, while Module (v) interfaces structured aviation data with the LLM.
  • III. METHODOLOGY: The framework uses CatBoost as its statistical expert and connects structured aviation data to the LLM through prompt construction and invocation.The figure explicitly identifies the Statistical Expert as the CatBoost model.
  • III. METHODOLOGY: The subsequent methodology sections detail the implementation of the five FlightLLM modules.The framework description presents the modules before their implementation is explained in later sections.

A. Data Preprocessing and Feature Engineering

The preprocessing pipeline converts high-dimensional QAR time series into an LLM-compatible representation centered on the 30-second pre-touchdown observation window. It combines statistically selected features with aviation-informed physical indicators to support discriminative and interpretable hard-landing analysis.

  • Hard landing definition: Hard landing labels indicate whether predicted VRTG exceeds an aircraft-type-specific safety threshold, using parameters including PITCH, ALT STD, and TLA.Samples below the threshold are classified as normal landings.
  • Data preprocessing: QAR data are preprocessed because their high-dimensional, multivariate time-series structure, varying sampling frequencies, and volume hinder direct LLM processing.The observation window spans 30 seconds before touchdown through touchdown.
  • Statistical feature extraction: TSFresh extracts candidate statistical features, which are grouped by sensor and filtered through significance tests to retain the top-k features.This branch targets statistically important features while controlling complexity and dimensionality.
  • Physical feature extraction: Aviation-domain indicators, such as minimum descent rate from 20 feet to touchdown, complement abstract statistical features with physically interpretable information.The physical feature branch is introduced to improve semantic clarity for human and LLM reasoning.

B. Semantic Discretization

The Semantic Discretization module converts continuous flight features into qualitative, distribution-based descriptions to address LLMs’ weak handling of raw numerical data. Each descriptor combines physical meaning, a quantile-derived semantic label, and the original numerical value to support flight-condition assessment and causal reasoning.

  • B. Semantic Discretization: LLM tokenization can fragment multi-digit numbers, disrupting numerical continuity and weakening their physical meaning.LLM tokenizers are designed for natural language rather than continuous numerical values.
  • B. Semantic Discretization: Semantic Discretization maps continuous flight features to qualitative expressions derived from their statistical distributions.It uses quantization thresholds at the 5th, 25th, 75th, and 95th percentiles.
  • B. Semantic Discretization: Each feature descriptor combines its physical meaning, quantile-based semantic label, and raw numerical value.The resulting hybrid expression preserves both qualitative and quantitative information.
  • B. Semantic Discretization: Predefined distributional abnormality levels reduce implicit numerical comparison and let the LLM focus on attribution reasoning.The semantic label provides an initial conceptual assessment of flight condition before processing the raw value.

C. Statistical Expert Hinting · D. Dynamic Context Retrieval

FlightLLM uses CatBoost as a statistical expert to provide prior guidance to the LLM, while dynamically retrieving similar flight instances to construct contrastive few-shot contexts. These components address limited labeled data and support more stable, context-informed reasoning for flight-safety analysis.

  • C. Statistical Expert Hinting: CatBoost is introduced as an auxiliary statistical expert that provides structured prior guidance to improve the stability and reliability of LLM outputs.The weak-to-strong design uses a smaller model to guide a larger model rather than replacing its reasoning process.
  • C. Statistical Expert Hinting: CatBoost transforms categorical features into numerical representations instead of relying on one-hot encoding, reducing information loss during boosting.Its encoding uses a prior value P and a prior weight β.
  • C. Statistical Expert Hinting: CatBoost also greedily combines tree-selected features as categories after the first split, enabling it to capture more complex dependencies.The passage notes empirical evidence that CatBoost often performs better than XGBoost.
  • C. Statistical Expert Hinting: CatBoost predictions serve as statistical anchors, with its predicted label and associated probability injected into the prompt as auxiliary contextual information.The LLM can use the signal without blindly following the CatBoost prediction.
  • D. Dynamic Context Retrieval: Large-scale fine-tuning is impractical for rare flight-safety events such as hard landings because collecting sufficient labeled data is difficult and costly.Traditional time-series classification methods also face difficulties adapting to new scenarios.
  • D. Dynamic Context Retrieval: For each query sample, the method dynamically retrieves similar historical instances from the dataset using cosine similarity in vector space.The similarity score compares the query feature vector with candidate sample feature vectors.
  • D. Dynamic Context Retrieval: The method selects the most similar normal-landing and hard-landing samples as references for contrastive reasoning.These retrieved samples are injected into the prompt as few-shot examples, allowing the LLM to compare subtle differences and identify key factors.

E. Prompt Construction and LLM Invocation

The prompt construction addresses aviation-specific semantic gaps by explicitly describing extracted features and guiding the LLM through structured, step-by-step reasoning. Chain-of-Thought prompting aligns inference with flight-safety specialists’ analytical workflow and supports final hard-landing judgments.

  • Feature Description Engineering: Feature description engineering provides explicit textual explanations to reduce misunderstandings of extracted features and their relationships with aircraft maneuvers.The method targets semantic gaps caused by insufficient knowledge of aviation dynamics, flight mechanics, and pilot control logic.
  • Chain-of-Thought Prompting: Chain-of-Thought prompting guides the LLM to analyze flight parameters step by step and interpret their operational significance.The prompt explicitly guides the generation of intermediate reasoning steps to enhance analytical performance.
  • Structured Reasoning: The structured reasoning process evaluates consistency with hard-landing mechanisms before deriving a final judgment.This workflow compensates for limited inherent aviation expertise and aligns inference with flight-safety specialists’ analysis.

IV. EXPERIMENTS

The experiments evaluate FlightLLM on hard landing, a high-risk event with complex, strongly coupled causal mechanisms, using a leakage-resistant pre-touchdown data segment. This benchmark tests both classification capability and attribution quality while preserving causal consistency.

  • Experimental Benchmark: Hard landing is selected as the experimental anchor because its complex, strongly coupled causes rigorously test FlightLLM’s classification and attribution capabilities.The study identifies hard landing as a representative high-risk flight safety event and a challenging benchmark.
  • Data Segmentation: The evaluation uses data from 30 seconds before touchdown through complete touchdown, defined as the moment when all landing gears touch ground.This dynamic endpoint is denoted t and determines each flight’s analysis interval.
  • Data Segmentation: The segmentation prevents post-touchdown signals from influencing cause analysis, thereby preserving causal consistency and avoiding data leakage.Signals after touchdown are excluded from the evaluation input.

A. Dataset and Experimental Configuration · B. Baselines

The experiments use 704 real-world A320 flights with standardized 4Hz trajectories and evaluate FlightLLM against established sequence, classical, convolutional, and interpretable baselines. The configuration includes three LLMs and four classification metrics, while baselines apply varied temporal, feature-selection, balancing, similarity, and attention mechanisms.

  • A. Dataset and Experimental Configuration: 704 flights comprise 282 hard landing and 422 normal landing samples, producing an approximately 2:3 hard-to-normal ratio.The samples were identified from 37,929 real-world A320 flight records and randomly selected to create a relatively balanced dataset.
  • A. Dataset and Experimental Configuration: Each flight is stored as a CSV trajectory containing 32 flight parameters, with all channels resampled to a standardized 4Hz sampling rate.The trajectories span engine start to shutdown, addressing inconsistent sampling frequencies across multiple channels.
  • A. Dataset and Experimental Configuration: GPT-3.5, DeepSeek-V1, and GLM-4.7-flash are evaluated using Accuracy, Precision, Recall, and F1-Score.These metrics are used to assess the performance of all selected large language models.
  • B. Baselines: The baseline set includes LSTM, SVM, RF, KNN, CNN, IMTCN, and SDTAN for comparison with FlightLLM.The baselines span conventional classifiers, deep temporal models, and interpretable or attention-based architectures.
  • B. Baselines: LSTM downsamples QAR parameters to 1Hz and uses memory blocks, gates, and peephole connections to model long-term sequential dependencies.The model is then employed for classification of sequential flight data.
  • B. Baselines: SVM uses height-based flight-data segments with Recursive Feature Elimination and grid search for feature selection and hyperparameter optimization.The resulting classifier processes sliced flight data segments.
  • B. Baselines: RF balances height-based segments with SMOTE before training a decision-tree ensemble, while KNN classifies fixed-length windows by nearest-neighbor majority vote.These baselines use distinct preprocessing and decision mechanisms for hard landing classification.
  • B. Baselines: CNN learns temporal features through convolution and pooling before MLP classification; IMTCN uses CAM for interpretability, and SDTAN models dynamic parameter relationships with interval attention.SDTAN combines an STG encoder, variable correlation extraction, and variable selection modules.

V. RESULTS AND DISCUSSION · A. Classification Performance · B. Interpretability Analysis

FlightLLM achieves competitive hard-landing classification performance and produces explanations consistent with flight-trajectory evidence. Its analysis distinguishes primary pilot-control factors from secondary environmental contributors and yields actionable flare guidance.

  • A. Classification Performance: FlightLLM-GPT and FlightLLM-Deepseek each achieve 81.56 Accuracy, exceeding SVM, RF, IMTCN, and SDTAN.The comparison covers three FlightLLM backbone models and seven baseline methods.
  • A. Classification Performance: CNN achieves the highest recall, while SDTAN attains the highest F1 score among the compared methods.The passage attributes their performance partly to convolutional operations, shared weights, fewer parameters, and improved generalization on small datasets.
  • B. Interpretability Analysis: The correctly classified hard-landing case is evaluated against normal-landing parameter distributions from 37,929 real-world A320 flight records.This statistical reference supports practical attribution analysis by comparing the case with historical normal landings.
  • B. Interpretability Analysis: During final approach, ALT QNH exceeds the 95th percentile, WIN SPD remains in the 80th–90th percentile range, and IVV exceeds the 95th percentile.IAS briefly falls below the 10th percentile at approximately 18 s before recovering, indicating short-term instability rather than sustained low energy.
  • B. Interpretability Analysis: FlightLLM identifies delayed flare timing and rapid pitch correction, aligning its reasoning chain with visual trajectory analyses and distinguishing main from secondary factors.The model’s interpretations include delayed or omitted flare and abrupt pitch maneuvers during the final phase.
  • B. Interpretability Analysis: Primary attributions emphasize pilot-control features, including pitch and vertical-speed behavior, while WIN SPD and IAS are treated as secondary environmental contributors.The model avoids assigning equal importance to every statistical deviation.
  • B. Interpretability Analysis: Counterfactual reasoning recommends initiating the flare earlier and maintaining a consistent pitch attitude to support smoother landing outcomes.The analysis links feature-level reasoning with raw QAR evidence and actionable training guidance.
  • B. Interpretability Analysis: The case exhibits 2.25 seconds from 20 ft to touchdown, 2.8 deg/s maximum pitch rate, and 41.84 maximum absolute pitch-command rate.These values are interpreted as limited flare time, abrupt pitch maneuvers, and drastic control-command changes.

C. Ablation Experiment

The ablation study shows that Semantic Discretization and Statistical Expert Hinting materially support FlightLLM’s classification performance. However, Semantic Discretization and Dynamic Context Retrieval can cause conservative or convergent decisions on borderline hazardous samples.

  • Module contributions: Removing Semantic Discretization drops Precision from 0.8571 to 0.6716, indicating that qualitative semantic labels help the LLM reason over numerical flight patterns.The labels provide cognitive anchors and an explicitly defined semantic scale instead of relying on implicit number comparison.
  • Module contributions: Removing Statistical Expert Hinting reduces Accuracy to 0.7518 and produces the lowest F1-Score among variants, demonstrating the value of CatBoost’s probabilistic guidance.CatBoost prediction probabilities constrain the LLM’s reasoning space.
  • Module contributions: Semantic Discretization and Dynamic Context Retrieval significantly improve classification accuracy but also cause cognitive convergence in some decisions.The ablation variants remove one module at a time while keeping the remaining components unchanged.
  • Limitations: Semantic Discretization may miss borderline high-risk samples when values fall just below a threshold and receive a less severe semantic label.A value just below the “Extremely High” boundary may be labeled “Slightly High,” leading the LLM to confidently classify the sample as safe.
  • Limitations: Dynamic Context Retrieval becomes more conservative on previously unseen borderline hazardous patterns, leading to lower recall.This behavior reflects the module’s cognitive convergence on unfamiliar borderline cases.

D. Consistency Analysis

The three LLMs show consistent feature-level attribution, with all assigning the greatest importance to IVV, PITCH, and time. These features correspond to abnormal vertical speed, pitch angle, and flare time, the three primary causes of hard landing.

  • D. Consistency Analysis: All three LLMs consistently assign the highest importance to IVV, PITCH, and time, aligning their feature-level attributions.The alignment is shown in a heatmap of feature distributions across models.
  • D. Consistency Analysis: IVV, PITCH, and time correspond directly to abnormal vertical speed, pitch angle, and flare time, respectively, as the three primary hard-landing causes.

VI. CONCLUSION

FlightLLM is proposed for interpretable flight safety event analysis, bridging QAR time-series data and text-based LLM reasoning. It combines hybrid feature engineering and domain knowledge injection to support classification and causal explanations, while remaining limited by the absence of aviation-specific fine-tuning and the need to modify prompts for additional events.

  • Contributions: FlightLLM bridges QAR time-series data and text data for LLM-based flight safety event analysis, targeting classification and causal explanations of hard landings.The approach is designed to address limited task-specific data and inject aviation domain knowledge into LLM inference.
  • Contributions: The Feature Engineering module converts raw QAR data into hybrid feature vectors combining statistical descriptors with physically meaningful flight indicators.This representation is intended to make flight data more compatible with language-based reasoning.
  • Limitations: The publicly available pre-trained language models were not fine-tuned specifically for hard landing, which may limit reasoning consistency and performance.The paper notes that aviation-focused fine-tuning could further improve both outcomes, despite the generalization capability of the current approach.
  • Limitations: Extending FlightLLM to multiple flight safety events requires modifying the prompts.The limitation concerns adapting the prompting strategy beyond hard landing analysis.
Loading 2608.18017v1…