Source-linked AI summary
Real-Time Anomaly Detection and Reactive Planning with Large Language Models
Rohan Sinha, Amine Elhafsi, Christopher Agia, Matthew Foutter, Edward Schmerling, Marco Pavone
TL;DR
Robotic systems can face rare out-of-distribution conditions, while LLM deployment is constrained by inference cost and the need to integrate judgments into safe control. AESOP combines fast embedding-based anomaly detection, slower generative hazard reasoning, and latency-aware MPC; the authors report favorable detection against zero-shot generative reasoning and feasibility in real-time agile-robot control.
Problem
Finite datasets may omit rare real-world conditions, while FM inference cost and safe integration of anomaly judgments complicate online use.
Source: §I · p. 1Method
AESOP uses prior-experience embeddings for fast anomaly detection, generative LLM reasoning for hazard assessment, and MPC maintaining feasible safety plans during reasoning latency.
Source: §I · p. 2Results
Embedding-based anomaly detection performs favorably to zero-shot generative reasoning with high-capacity LLMs, and dual-stage reasoning supports real-time reactive control of agile robots.
Source: §VI · p. 10Takeaways & Limitations
The experiments support practical real-time integration of foundation-model reasoning in dynamic robotic systems, including onboard use of small language models.
Source: §I · p. 2, §VI · p. 10
Abstract
from arXiv · showhide
Foundation models, e.g., large language models (LLMs), trained on internet-scale data possess zero-shot generalization capabilities that make them a promising technology towards detecting and mitigating out-of-distribution failure modes of robotic systems. Fully realizing this promise, however, poses two challenges: (i) mitigating the considerable computational expense of these models such that they may be applied online, and (ii) incorporating their judgement regarding potential anomalies into a safe control framework. In this work, we present a two-stage reasoning framework: First is a fast binary anomaly classifier that analyzes observations in an LLM embedding space, which may then trigger a slower fallback selection stage that utilizes the reasoning capabilities of generative LLMs. These stages correspond to branch points in a model predictive control strategy that maintains the joint feasibility of continuing along various fallback plans to account for the slow reasoner's latency as soon as an anomaly is detected, thus ensuring safety. We show that our fast anomaly classifier outperforms autoregressive reasoning with state-of-the-art GPT models, even when instantiated with relatively small language models. This enables our runtime monitor to improve the trustworthiness of dynamic robotic systems, such as quadrotors or autonomous vehicles, under resource and time constraints. Videos illustrating our approach in both simulation and real-world experiments are available on this project page: https://sites.google.com/view/aesop-llm.
I. INTRODUCTION
Robots may encounter rare conditions absent from finite training data, motivating runtime monitoring and safety interventions; AESOP pairs fast anomaly detection with slower hazard reasoning and latency-aware planning. The paper frames this against prior work with limited dynamic reactivity and formalizes semantic hazards that ordinary state constraints may not capture.
I. INTRODUCTION
- Finite development datasets may miss rare real-world conditions, so runtime monitors must assess reliability and enact safety-preserving interventions when needed.The paper describes unexpected failures when conditions deviate from training data or rare situations were underrepresented at design time.Source: §I · p. 1
- AESOP combines embedding-based fast anomaly detection, slower generative hazard reasoning, and MPC that maintains jointly feasible safety-preserving plans during reasoning latency.The framework uses an upper bound on the slow reasoner’s completion time in its control design.Source: §I · p. 2
- AESOP’s evaluation spans text domains, simulated and real-world quadrotor experiments, and CARLA recreations of autonomous-vehicle failures.The authors conclude that real-time integration in dynamic, agile robotic systems is practically feasible.Source: §I · p. 2
II. RELATED WORK
- Prior FM robotics work often assumes quasi-static or offline operation, leaving response-time demands and dynamic reactivity insufficiently addressed for agile robots.The related-work discussion notes that delayed responses can leave fast-moving robots in situations where a crash becomes unavoidable.Source: §II · p. 3
III. PROBLEM FORMULATION
- Semantic failure modes depend on qualitative task context beyond precise state constraints, such as landing on a designated zone covered in burning debris.The formulation assumes nominal observations and control-invariant recovery regions corresponding to high-level safety interventions.Source: §III · p. 3
IV. PROPOSED APPROACH
AESOP uses prior nominal experiences to ground an embedding-based fast anomaly detector, then invokes a slower generative reasoner to assess hazards and select interventions. The approach accounts for that reasoner’s latency in the runtime-monitoring design.
A. Runtime Monitor: Fast and Slow Reasoning
- AESOP separates anomaly detection from slower reasoning about an anomaly’s downstream consequence and whether a safety-preserving intervention is needed.The fast stage flags deviations from nominal experience; the generative stage assesses whether intervention is appropriate.Source: A. Runtime Monitor: Fast and Slow… · p. 3
- The fast detector embeds prior nominal observations offline, embeds each runtime observation, and scores it against the cached experience vectors.The simplest example score uses maximum cosine similarity to retrieve the most similar prior experience.Source: A. Runtime Monitor: Fast and Slow… · p. 4
- The anomaly threshold τ is calibrated as an α-quantile of nominal scores, with leave-one-out scoring for nominal embeddings.The paper identifies precise false-positive or false-negative guarantees via conformal prediction as a possible future extension.Source: A. Runtime Monitor: Fast and Slow… · p. 4
- After an anomaly is detected, the slow LLM generates a zero-shot assessment of whether to engage interventions associated with recovery sets.Its output can specify continued nominal operation or an intervention and corresponding recovery set.Source: A. Runtime Monitor: Fast and Slow… · p. 4
- The design assumes a known upper bound K on the slow reasoner’s response time, which can be estimated from constrained output lengths or network field tests.Latency may arise whether inference runs onboard or remotely over unreliable networks.Source: A. Runtime Monitor: Fast and Slow… · p. 4
B. Planning a Tree of Recovery Trajectories
The MPC jointly plans a nominal trajectory and recovery trajectories, keeping recovery options feasible while the slow reasoner decides. Theorem 1 states the resulting constraint and recovery guarantees, assuming the MPC is initially feasible.
B. Planning a Tree of Recovery Trajectories
- The MPC optimizes a nominal trajectory alongside recovery trajectories that reach their respective recovery sets within horizon T.The nominal plan minimizes the control objective; the recovery plans preserve safety interventions for the slow reasoner.Source: B. Planning a Tree of Recovery… · p. 4
- Consensus constraints keep nominal and recovery inputs aligned initially, then keep recovery trajectories aligned for K timesteps to preserve feasible options during reasoning latency.The fast detector triggers both the first K recovery actions and the slower reasoning, so the available options remain feasible when the reasoner responds.Source: B. Planning a Tree of Recovery… · pp. 4–5
- Theorem 1 guarantees state and input constraints, at least one feasible safety intervention at every time, and arrival at a selected recovery set within T + 1 timesteps after the slow reasoner chooses it.These guarantees assume the MPC is initially feasible for some set of recovery strategies.Source: B. Planning a Tree of Recovery… · p. 5
- After an anomaly warning, the robot balances nominal progress with preserving fallback feasibility, generally slowing down until the reasoner selects a response.It then returns to nominal operation or commits to the selected intervention; at least one option remains feasible, though multiple options are not explicitly guaranteed throughout nominal operation.Source: B. Planning a Tree of Recovery… · p. 5
V. EXPERIMENTS
The experiments test fast embedding-based detection, generative anomaly assessment, and integration with reactive planning in simulated and real robotic settings. Their synthetic domains represent anomalies as atypical observations relative to a robot’s nominal experience, with complexity ranging from manipulation to VTOL.
V. EXPERIMENTS
- The experiment program tests embedding-based detection, generative assessment, integrated drone control in simulation and on hardware, and transfer to a semantically rich self-driving environment.The hypotheses cover detector performance and model size, generative assessment of detected anomalies, real-time agile control, and language or multimodal embeddings for detection.Source: §V · p. 5
A. Synthetics—Manipulation, Autonomous Vehicles, VTOL
- The three synthetic domains model observations as collections of concepts, and define anomalies as deviations from task-specific nominal experience, whether or not they pose safety risks.Concepts can combine objects and semantic relationships; the domains are warehouse manipulation, autonomous driving, and VTOL flight or landing.Source: A. Synthetics—Manipulation, Autonomous… · p. 6
- The synthetic tasks include hazardous objects or combinations on a conveyor, unusual objects or combinations on roads, and unexpected flying objects or landing zones.Examples include a leaking bleach bottle, a blank speed limit sign, and a rooftop on fire.Source: A. Synthetics—Manipulation, Autonomous… · p. 6
- The synthetic domains increase in complexity from warehouse manipulation to autonomous vehicles to VTOL, where anomalous concepts can resemble nominal ones.For example, a flying bird and a swarming flock of birds are conceptually similar, although only the latter is anomalous in the stated example.Source: A. Synthetics—Manipulation, Autonomous… · p. 6
1) Fast Reasoning for Anomaly Detection (H1, H2):
The fast detector uses language-model embeddings and prior nominal observations to identify anomalies, while generative models assess whether detected anomalies warrant intervention. Across the reported experiments, embedding detection outperforms GPT-4 generative baselines when concept coverage is sufficient, and chain-of-thought reasoning improves slow-assessment accuracy over single-token prediction.
1) Fast Reasoning for Anomaly Detection (H1, H2):
- The fast anomaly detector was evaluated with nine language models, spanning BERT-base (110M) to Mistral (7.11B) and including three OpenAI embedding models; reported results use top-5 scoring.Generative baselines were GPT-4 single-token prediction and chain-of-thought reasoning. The detection threshold was the nominal dataset’s 95th quantile.Source: 1) Fast Reasoning for Anomaly… · p. 6
- With sufficient concept coverage, the fast detector outperforms GPT-4 single-token and chain-of-thought anomaly-classification baselines.Accuracy rises logarithmically with sampled experience size but linearly with concept coverage; Fig. 2 presents accuracy across the manipulation, autonomous-vehicle, and VTOL domains.Source: 1) Fast Reasoning for Anomaly… · p. 6, Figure 2 · p. 7
- Sentence Transformer MPNet (110M) often outperforms BERT Large (336M) and OpenAI embedding models, and performs comparably to Llama 2 (7B) and Mistral (7.11B).The authors report that model performance does not strictly correlate with size, but depends on training data and strategy.Source: 1) Fast Reasoning for Anomaly… · p. 6
- Generative baselines are difficult to ground in prior experience because of limited context windows and challenges such as recency bias.The embedding detector instead grounds observation analysis in previously observed concepts.Source: 1) Fast Reasoning for Anomaly… · p. 6
2) Slow Reasoning for Anomaly Assessment (H3):
- For VTOL safety assessment, GPT variants outperform the Llama 2 (7B) baseline, while chain-of-thought reasoning improves overall accuracy (11% for GPT-4) and reduces false positives.A true positive is correctly engaging a fallback for a safety-concerning anomaly; a true negative is correctly dismissing a safety-redundant anomaly.Source: 2) Slow Reasoning for Anomaly… · p. 7, Table I · p. 7
B. Full Stack—Quadrotor Simulation (H4)
The quadrotor simulation evaluates latency-aware recovery planning, while hardware demonstrations test nominal, consequential, and inconsequential anomalies and measure onboard inference costs. The results describe feasible recovery behavior at dynamic speeds and show that the embedding pipeline can support approximately 10 Hz operation.
B. Full Stack—Quadrotor Simulation (H4)
- Two recovery plans reach their respective regions while preserving nominal progress and remain aligned for K timesteps as the quadrotor descends and slows.The shared initial segment budgets time for the slow reasoner to select a recovery plan.Source: B. Full Stack—Quadrotor Simulation (H4) · p. 8, Figure 3 · p. 8
- The latency-aware FSMPC mildly improves over the naive post-hoc baseline, and the quadrotor averages around 2.5m/s during the closed-loop trajectory.Table II compares recovery success across 500 scenarios, including against fallback planning that ignores runtime-monitor latency.Source: B. Full Stack—Quadrotor Simulation (H4) · p. 8, Table II · p. 8
C. Quadrotor Hardware Demonstration & Timing (H4)
- For the hardware embedding cache, OWL-ViT descriptions of visible objects provide context for MPNet prompts and GPT-3.5-Turbo reasoning.The observations are recorded while flying a circular path above the operating area with nominal ground clutter.Source: C. Quadrotor Hardware Demonstration &… · p. 9
- In hardware scenarios, the quadrotor lands normally, diverts to an alternative landing site or holding zone when landing boxes are occupied, and ignores an inconsequential keyboard anomaly.The hardware setup uses a downward-facing camera in a cluttered environment.Source: Figure 4 · p. 8, C. Quadrotor Hardware Demonstration &… · p. 9
- 18.8Hz is the joint rate for OWL-ViT detection parsing and MPNet embedding computation, supporting AESOP operation at approximately 10Hz on a Jetson AGX Orin.The cloud-based GPT-3.5-Turbo slow reasoner has non-negligible latency.Source: C. Quadrotor Hardware Demonstration &… · p. 9, Table III · p. 9
D. Ablation—Autonomous Vehicles Simulation
The CARLA ablations compare reasoning and embedding-based anomaly detection using text and vision observations. The results report benefits from chain-of-thought reasoning and stronger embedding performance in some settings, while highlighting calibration-related limits for CLIP.
D. Ablation—Autonomous Vehicles Simulation
- The CARLA study evaluates text and image-based detection, including OWL-ViT-derived prompts, language embeddings, and CLIP, with ground-truth detections used for a fair text-versus-vision comparison.Its simulated anomalies are based on documented autonomous-vehicle perception failures, and the study also considers single-token and chain-of-thought reasoning.Source: D. Ablation—Autonomous Vehicles… · p. 9, 1) End-to-End Reasoning for Anomaly… · p. 9, 2) End-to-end Embedding-Based Anomaly… · pp. 9–10
1) End-to-End Reasoning for Anomaly Detection (H5):
- Chain-of-thought reasoning yields more accurate responses than single-token reasoning in text- and vision-based anomaly detection, while GPT-4V results suggest end-to-end detection may be possible.The comparison is reported in Table IV.Source: 1) End-to-End Reasoning for Anomaly… · p. 9
2) End-to-end Embedding-Based Anomaly Detection (H5):
- MPNet is less capable than Mistral at identifying anomalies among many nominal observations, and its accuracy degrades as object counts increase while Mistral’s remains consistent.Mistral also surpasses GPT-4 and GPT-4V, though it is roughly 64x larger than MPNet.Source: 2) End-to-end Embedding-Based Anomaly… · p. 10
- CLIP embeddings achieve performance comparable to GPT-4V CoT, but CLIP’s false-positive rate increases significantly when calibrated only on nominal observations from routes without anomalies.The authors attribute this limitation to visual novelty and call for further work to disentangle visual and semantic features.Source: 2) End-to-end Embedding-Based Anomaly… · p. 10
VI. CONCLUSION AND OUTLOOK
- Embedding-based anomaly detection performs favorably to zero-shot generative reasoning, while vision-based embeddings can detect anomalies in high-dimensional observations.The conclusion also reports that small language models can support onboard use and dual-stage reasoning can operate in an agile robot’s real-time control loop.Source: §VI · p. 10
- Further analysis is needed on whether the LLM selects correct fallback plans and whether a programmatic fallback can be determined after a latency timeout.The conclusion also proposes constrained word budgets, intermediate generations, and continual learning as future directions.Source: §VI · p. 10
APPENDIX
This appendix supplies notation and anomaly-detection background before evaluating embedding detectors across synthetic robotics domains. Its ROC analysis finds that MPNet closely rivals larger models, with concept coverage especially important in the more complex VTOL domain.
APPENDIX
- The appendices provide experimental details, additional results and ablations, analyses, and proofs of theoretical results.They also point readers to quadrotor experiment videos and project-page materials.Source: APPENDIX · p. 14
A. Notation and Glossary
- The notation glossary collects the paper’s symbols and conventions, including indexed time-varying quantities and predicted quantities at future timesteps.The full glossary is provided in Table V.Source: Table V · p. 14, A. Notation and Glossary · p. 14
B. Background: Anomaly Detection
- Anomaly detection constructs a scalar score, then calibrates a threshold so observations above it are anomalous and those at or below it are nominal.The appendix notes that scarce anomalous examples typically preclude supervised learning with an obvious decision boundary.Source: B. Background: Anomaly Detection · p. 15
C. Background: Mechanics of LLMs
- Autoregressive LLM reasoning generates and appends next-token outputs, incurring computational cost that scales superlinearly relative to computing an embedding.Direct learning on embeddings instead requires a source of supervision.Source: C. Background: Mechanics of LLMs · p. 15
D. Additional Results: Synthetics
- The synthetic experiments assess detector quality with AUROC, sensitivity to detection thresholds, and sensitivity to the similarity score function.The evaluation spans robotics domains with diverse observational and task semantics.Source: D. Additional Results: Synthetics · p. 15
1) Fast Anomaly Detector ROC Analysis:
- MPNet closely rivals 7B models on ROC performance, while concept coverage becomes especially important in the more complex VTOL domain.The comparison uses AUROC across sampled experiences and accuracy across sampled concepts.Source: 1) Fast Anomaly Detector ROC Analysis · p. 15
2) Detection Threshold Analysis:
The additional results examine threshold and similarity-score choices, manipulation-domain safety assessment, and quadrotor implementation. Across threshold settings, MPNet and Mistral generally perform similarly and outperform GPT-4 reasoning, while the quadrotor implementation runs in real time.
2) Detection Threshold Analysis:
- MPNet performs nearly identically to Mistral (7B) across thresholds, and both consistently outperform GPT-4 reasoning except at the 75th quantile.Accuracy increases with threshold quantile; results are reported in Table VI.Source: 2) Detection Threshold Analysis · pp. 15–16, Table VI · p. 17
3) Similarity Score Function Analysis:
- Top-k scores with k in {3,5,10} are competitive across data regimes, while top-1 and Mahalanobis perform poorly in low-data settings for most models.At full concept coverage, score-function accuracy differences are within approximately 5% across evaluated models except OpenAI Ada 002.Source: 3) Similarity Score Function Analysis · p. 16, Figure 6 · p. 17
4) Safety Assessment in Manipulation Domain:
- Safety assessment is easier in manipulation than in VTOL, and GPT-4 gains strongly over GPT-3.5 at recognizing inconsequential anomalies.Table VII reports the manipulation-domain assessment.Source: 4) Safety Assessment in Manipulation… · p. 16, Table VII · p. 17
1) Simulation and Implementation Details:
- The quadrotor simulation constrains speed to 1.5m/s per principal axis and assumes the slow LLM reasoner takes at most K =1.5s to decide.It uses four recovery regions and plans over T =4s with dt=.1s.Source: 1) Simulation and Implementation Details · p. 17
- AESOP’s MPC implementation runs at approximately 42Hz on an Nvidia Jetson Orin AGX CPU, supporting real-time operation with the fast detector.The implementation solves multiple MPC variants for combinations of recovery sets and selects the feasible plan with least cost.Source: 1) Simulation and Implementation Details · pp. 17–18
2) Baselines:
The quadrotor comparisons show why recovery planning must account for the slow reasoner’s latency: maintaining feasible recovery plans alone does not ensure safe recovery. AESOP also handles inconsequential anomalies with little disruption, and the autonomous-vehicle ablations identify perception and scene-complexity sensitivities.
2) Baselines:
- Unlike naive MPC and Fallback-Safe MPC, AESOP maintains recovery options that remain usable while the LLM decides, enabling safe recovery after delays.The baselines either do not maintain feasible recovery plans or do not account for the slow reasoner’s latency.Source: 2) Baselines · p. 18
3) Results:
- AESOP minimally impedes nominal goal completion and can resume nominal flight when the LLM judges a detected anomaly inconsequential.The qualitative examples also show safe recovery when an anomaly is consequential to safety.Source: 3) Results · p. 18
- AESOP reaches the LLM-selected recovery region in every one of 500 randomized consequential-anomaly scenarios, while accounting for LLM latency is essential for reliability.FSMPC improves over naive MPC but does not match AESOP’s latency-aware recovery; Table VIII reports recovery percentages.Source: 3) Results · p. 18, Table VIII · p. 18
F. Additional Results: Autonomous Vehicle Simulation
- The autonomous-vehicle ablations find that ground-truth scene descriptions help overcome detector errors, while larger models are needed for semantically rich scenes with many objects.The two-stage language-embedding pipeline misses visual novelty, whereas multimodal CLIP can produce false positives on visually novel but semantically nominal observations.Source: F. Additional Results: Autonomous… · p. 20
1) Object Detection vs. Ground-truth Detections:
In CARLA, language-embedding detection depends on reliable scene descriptions and model capacity: ground-truth detections substantially improve results, while raw detector errors obscure semantic differences. The ablations also show sensitivity to route novelty and scene complexity.
1) Object Detection vs. Ground-truth Detections:
- With ground-truth detections, Mistral (7B) reaches 0.94 mean accuracy, surpassing GPT-4V CoT by 4% absolute; raw scene descriptions yield at best 0.70 mean accuracy.The ground-truth processing removes false positives and inserts missed true detections, so this is a proof of concept rather than a deployable preprocessing step.Source: 1) Object Detection vs. Ground-truth… · pp. 20–21, Figure 10 · p. 21
2) Calibration Ablation on Withheld Trajectories:
- When nominal data from anomaly-containing routes is withheld, CLIP’s false-positive rate increases, whereas the two-stage language pipeline avoids that performance drop.The authors attribute CLIP’s false positives to visual novelty in unseen routes and suggest balancing visual and semantic features in future multimodal detectors.Source: Table IX · p. 21, 2) Calibration Ablation on Withheld… · p. 21
3) Accuracy vs. Complexity of Observations:
- MPNet accuracy drops as the number of objects in an observation increases, while its renewed similarity to Mistral at 7–8 objects may reflect class imbalance and fewer such observations.The results suggest larger models are needed for anomalousness judgments in complex scenes with many objects.Source: Figure 11 · p. 21, 3) Accuracy vs. Complexity of… · p. 21, Figure 12 · p. 22
G. Prompting Details
- Language-based prompts provide the monitored robot and task context before the current task and observation, because object labels alone failed to capture their relation to the task and other objects.The prompt ends with an instruction specifying the monitor’s output.Source: G. Prompting Details · pp. 21–22
H. Proof of Theorem 1
Theorem 1 establishes that AESOP’s MPC remains safe and feasible when a slow reasoner is invoked, provided the initial MPC is feasible and recovery sets are control invariant.
H. Proof of Theorem 1
- The guarantee assumes control-invariant recovery sets, so states in each set can remain there under an admissible input.The proof invokes the standard definition of a control-invariant set and notes that landed quadrotor states are one example.Source: H. Proof of Theorem 1 · p. 22
- Given initial MPC feasibility, Theorem 1 guarantees state and input constraints, an available feasible safety intervention at every time, and eventual entry into a selected recovery set.The selected recovery set is reached by t_anom + T + 1 when the slow reasoner chooses an intervention.Source: H. Proof of Theorem 1 · p. 22
- Recursive feasibility during the reasoner’s delay follows by carrying feasible intervention plans forward while the prediction and consensus horizons shrink.The proof maintains feasibility through the reasoner’s return, then continues with its chosen recovery or nominal operation.Source: H. Proof of Theorem 1 · pp. 22–23
I. Hardware Experiment – Additional Details and Results
The hardware experiment collects nominal and calibration observations from quadrotor flights to build and evaluate an anomaly detector. The section also points readers to experiment videos.
I. Hardware Experiment – Additional Details and Results
- The hardware experiments include data collection, detector calibration, latency analysis, test scenarios, and MPC implementation details.Experiment videos are available in the supplementary materials and on the project page.Source: §I · p. 23
1) Data Collection:
- The quadrotor flies circularly above the operational area, and prompts formed from combinations of up to four detected objects are embedded into a vector cache.Nominal experience data contain no anomalous scene elements; a separate calibration set introduces unseen objects and varied placements.Source: 1) Data Collection · p. 23
- Figures 13 and 14 show examples of nominal and anomalous quadrotor observations, respectively.The anomalous example depicts an obstruction of the landing zone.Source: Figure 13 · p. 23, Figure 14 · p. 23, 1) Data Collection · p. 23
2) Fast Anomaly Detector Calibration:
The hardware results calibrate a top-1 anomaly detector and bound slow-reasoner latency, then describe nominal, consequential, and inconsequential anomaly scenarios. The controller uses a constrained MPC implementation.
2) Fast Anomaly Detector Calibration:
- Top-1 scoring performed best on the calibration set, and the detector threshold was set to achieve at least 0.9 TPR.The authors attribute top-1’s advantage to limited diversity of possible observations in the scene and use it in closed-loop evaluations.Source: 2) Fast Anomaly Detector Calibration · p. 23, Figure 15 · p. 24
3) Choosing K - Analysis of LLM Query Latencies:
- 4.3s was chosen as the slow-reasoner latency bound, corresponding to the 95% response-time quantile in 500 GPT-3.5-Turbo queries.Response times had a 3.1s mean and 0.85s standard deviation; no latency exceeded the bound in the experiments.Source: 3) Choosing K - Analysis of LLM Query… · pp. 23–24, Figure 16 · p. 24
2. Consequential Anomaly: We consider two variants
- With a landing-zone obstruction, the LLM selects the blue box when available and the holding zone when both landing boxes are occupied.The fast detector makes the quadrotor pull back toward recovery regions while the LLM decides; the consensus horizon keeps recovery plans identical until then.Source: §2 · p. 24
4) Summary of test scenarios:
- A previously unseen keyboard triggers the fast detector, but the LLM deems it inconsequential and the quadrotor resumes landing at its nominal site.The quadrotor slows while awaiting the decision, then speeds back up toward the landing zone.Source: §2 · p. 24
5) Controller Parameters:
- The MPC uses acceleration commands, a 10s horizon with 0.05s discretization, and velocity and position/altitude safety constraints.A kinematic drone model supplies trajectory predictions, which PX4 PID controllers track.Source: 5) Controller Parameters · p. 24