ALFA: A Dataset for UAV Fault and Anomaly Detection

• Learning-based methods can predict known faults with high precision but may struggle with new issues and depend on substantial training data.Source: §1 · p. 1
• Collecting real-aircraft fault data is costly and risky, while most proposed methods are tested in simulation and few use substantial real-flight data.The authors note that this makes comparisons between simulation-tested and real-flight-tested methods difficult.Source: §1 · p. 1
• ALFA contains processed data from 47 autonomous flights with eight types of sudden control-surface faults, including 23 full engine failures.It includes 66 minutes of normal flight and 13 minutes of post-fault flight time.Source: §1 · p. 1
• ALFA also includes hours of raw autonomous, autopilot-assisted, and manual flight data, with fault-time and fault-type ground truth for processed scenarios.The ground truth is provided to help evaluate new methods.Source: §1 · p. 1
• Helper codes in C++, Python, and MATLAB support working with the processed data and evaluating methods.Source: §1 · p. 2
• The precise robot dynamics model is not provided, so methods requiring it must estimate a model from data or combine general airplane dynamics with the hardware description.Source: §1 · p. 2

2.1 Hardware Setup

• The modified Carbon Z T-28 has a 2-meter wingspan, electric engine, flight-control surfaces, PX4 autopilot, Pitot tube, GPS, and Jetson TX2 computer.Source: §2.1 · p. 2

2.2 Software

• Custom ArduPlane firmware adds parameters to simulate engine failure, stuck elevator, rudder hardover, and stuck ailerons.The rudder can be fixed left, right, or middle; the ailerons can be fixed individually or together.Source: §2.2 · p. 2
• Fault parameters operate only in autonomous mode, and the safety pilot can take over at any time, restoring disabled actuators and the engine.Source: §2.2 · p. 2
• The onboard computer records flight and state information as a ROS bag, while a node publishes fault ground truth by checking the custom parameters.The autonomous trajectory controller is modified to run on the onboard TX2 computer.Source: §2.2 · p. 2
• Modified firmware and MAVROS publish internal autopilot commands, such as commanded roll and pitch, at high frequency over MAVLink.Source: §2.2 · p. 2

3 Dataset

• The dataset was collected at an airport near Pittsburgh, and each flight sequence may contain only part of the full trajectory.Recorded portions can be extracted from the data.Source: §3 · p. 2

3.1 Data Formats

• Processed autonomous sequences contain at most one fault each, while raw flight logs, TX2 telemetry, and Pixhawk logs provide additional unprocessed data.Processed sequences are available as CSV, MATLAB, and ROS bag files; the unprocessed telemetry and Dataflash logs omit fault ground truth and may suit unsupervised detection.Source: §3.1 · p. 3

3.2 Fault Types

• A large portion of the processed dataset concerns engine failure, alongside varied faults intended to encourage methods that handle multiple fault types.The passage does not provide the fault-type table’s cell values.Source: §3.2 · p. 3, Table 1 · p. 3

3.3 Data Description

• Fault ground-truth topics appear only during faults and are recorded at about 5 Hz, so the first message may lag the fault by up to 0.2 seconds.The sequences also include measured and commanded roll, pitch, velocity, airspeed, and yaw at 20–25 Hz.Source: §3.3 · p. 4

3.4 Using the Dataset

• Provided tools load and iterate through data, plot topic fields, separate normal from post-fault flight, and report false positives and detection delays for evaluated methods.Base tools are available in C++11, MATLAB, and Python 3.x; evaluation code returns additional statistics.Source: §3.4 · p. 4

4 Evaluation Metrics

• Metric choice depends on a method’s class, fault type, and application; detection delay can be critical online but less relevant or meaningless offline.For example, delay may be meaningless for fault classification methods.Source: §4 · p. 4
• Maximum detection time measures the fault-to-detection delay and is useful for online evaluation, where a large delay can lead to irreversible loss of vehicle control.The authors identify it as a useful metric for comparing online methods.Source: §4 · p. 5
• Average detection time summarizes a method’s overall detection-time performance across fault scenarios and is useful for evaluating online methods.Source: §4 · p. 5
• Accuracy uses the proportion of correctly classified sequences, while precision and recall reflect reliability among detections and in detecting fault-containing sequences, respectively.Accuracy is described as working best when false detections and false negatives have similar costs.Source: §4 · p. 5

• ALFA provides autonomous fixed-wing UAV flight data with scenarios of different control-surface faults occurring mid-flight.Source: §5 · p. 5
• The dataset is not complete: it includes sudden control-surface failures but not other possible faults such as sensor issues or gradual errors.The authors invite contributions covering other fault types and platforms.Source: §5 · p. 5
• The authors provided base codes for dataset use and evaluation, and identify creating a benchmark as a way to improve comparisons with state-of-the-art methods.Source: §5 · p. 5