PaperlayerView exact revision on arXiv

Source-linked AI summary

RAPTOR: A Foundation Policy for Quadrotor Control

Jonas Eschmann, Dario Albani, Giuseppe Loianno

arXiv:2509.11481v2cs.ROcs.AIcs.LG

TL;DR

Quadrotor control often requires retuning after hardware changes, while changing the environment can require full policy retraining. RAPTOR distills 1000 teacher policies into an adaptive student, which controls a broad range of platforms without retraining while approximately matching state-of-the-art solutions.

  • Problem

    Hardware changes can require controller retuning, and changing the environment can require full policy retraining, limiting broad adoption of quadrotor control methods.

    Source: Introduction · p. 3

  • Method

    RAPTOR uses Meta-Imitation Learning to distill 1000 Markovian teacher policies into a single adaptive, non-Markovian student policy.

    Source: Introduction · p. 5

  • Results

    RAPTOR controlled a broad range of quadrotors without retraining through in-context learning, approximately matching state-of-the-art end-to-end control while being computationally lighter.

    Source: Discussion · p. 16

  • Takeaways & Limitations

    The results support a single policy as a flexible alternative to platform-specific end-to-end control policies.

    Source: Discussion · p. 16

    Out-of-distribution generalization has limits: Flightmare tracking was lower at thrust-to-weight ratio ≈12 than within the training range of ≤5.

    Source: Discussion · p. 17

Abstract

from arXiv · show

Humans are remarkably data-efficient when adapting to new unseen conditions, like driving a new car. In contrast, modern robotic control systems, like neural network policies trained using Reinforcement Learning (RL), are highly specialized for single environments. Because of this overfitting, they are known to break down even under small differences like the Simulation-to-Reality (Sim2Real) gap and require system identification and retraining for even minimal changes to the system. In this work, we present RAPTOR, a method for training a highly adaptive foundation policy for quadrotor control. Our method enables training a single, end-to-end neural-network policy to control a wide variety of quadrotors. We test 10 different real quadrotors from 32 g to 2.4 kg that also differ in motor type (brushed vs. brushless), frame type (soft vs. rigid), propeller type (2/3/4-blade), and flight controller (PX4/Betaflight/Crazyflie/M5StampFly). We find that a tiny, three-layer policy with only 2084 parameters is sufficient for zero-shot adaptation to a wide variety of platforms. The adaptation through in-context learning is made possible by using a recurrence in the hidden layer. The policy is trained through our proposed Meta-Imitation Learning algorithm, where we sample 1000 quadrotors and train a teacher policy for each of them using RL. Subsequently, the 1000 teachers are distilled into a single, adaptive student policy. We find that within milliseconds, the resulting foundation policy adapts zero-shot to unseen quadrotors. We extensively test the capabilities of the foundation policy under numerous conditions (trajectory tracking, indoor/outdoor, wind disturbance, poking, different propellers).

Introduction

RAPTOR aims to adapt quadrotor control to unseen dynamics from high-frequency interaction, addressing policies that overfit individual platforms or trajectories and may need retraining after changes. It uses a recurrent policy trained across plausible quadrotor dynamics so it can infer system-relevant information implicitly while controlling the vehicle.

Motivation

  • Quadrotor controllers often need retuning after hardware changes, while RL policies can overfit a dynamics model or trajectory and domain-randomized stateless policies may act conservatively.Changing a propeller or battery can require retuning classical controllers; changing an environment can require full policy retraining.Source: Introduction · p. 3

Approach

  • RAPTOR uses a recurrent policy to infer dynamics relevant to control from interaction, training across a broad distribution of physically plausible quadrotors rather than reconstructing system parameters explicitly.The training objective is task reward; inferred quantities may include thrust-to-weight or torque-to-inertia ratios and aggregates of motor delays or thrust curves.Source: Introduction · pp. 3–4

Evaluation

  • The study evaluates RAPTOR across 10 real quadrotors and indoor and outdoor conditions, including unseen platforms, altered propellers, flight-time impacts, and a platform beyond the trained thrust-to-weight range.It also investigates policy size and inference speed, context-window extrapolation, activation-time system inference, and tracking from quasi-static to highly dynamic trajectories.Source: Introduction · p. 4

Contributions

  • The authors study Meta-Imitation Learning scaling laws and provide an implementation for resource-constrained hardware, plus means to use the policy across flight-controller firmwares and simulation environments.They also deploy the final policy directly on microcontrollers, including those of the tiniest quadrotors.Source: Introduction · pp. 4–5
  • Meta-Imitation Learning distills 1000 Markovian teacher policies into one adaptive, non-Markovian student, forming RAPTOR’s end-to-end foundation policy.The method also specifies a physically plausible dynamics distribution and a formal design for the RAPTOR architecture.Source: Introduction · p. 5

Related work

  • Unlike the adaptive policy in (23), RAPTOR is a full position controller rather than a CTBR-setpoint controller, and it uses 2084 parameters versus 114872 in (23).The approach in (6) is stateless and was demonstrated on two quadrotors; RAPTOR’s authors also distinguish their multi-teacher method from prior privileged-teacher distillation work.Source: Introduction · pp. 5–6

Results

RAPTOR produces a robust, adaptable quadrotor policy that works across diverse real and simulated platforms and conditions. Training results, emergent system identification, and deployment tests show adaptation to unseen dynamics and demanding tasks.

Pre-Training

  • All 1,000 teacher-policy training runs converged reliably, and good flight behavior appeared after 100,000 steps for most policies.The authors attribute stability to overparameterized teachers, reward tuning, and off-policy SAC with retained replay-buffer interactions.Source: Pre-Training · p. 7

Meta-Imitation Learning

  • After about 1,000 epochs, the student converged to good performance on all 7 unseen quadrotors spanning agile to non-agile and lightweight to heavy dynamics.Meta-Imitation Learning took 1.9 h, and the selected policy size required less than 10% of available computation on the most resource-constrained quadrotors.Source: Meta-Imitation Learning · pp. 8–9, Figure 2 · p. 8

Emergent System Identification

  • A linear probe predicted thrust-to-weight ratio with MSE 0.047 and R^2 0.949, showing emergent implicit system identification in the policy’s latent state.The estimate improved over time as the policy recovered from an initial state displaced 2 m in x and y and pitched backward 90°.Source: Emergent System Identification · p. 9, Figure 3 · p. 10

Deployment

  • The policy controlled 10 real quadrotors and transferred across platforms, including out-of-distribution conditions such as flexible frames and thrust-to-weight ratios above the training range.The authors report robust performance on all platforms, while noting low-frequency z-axis oscillations on some non-EKF-based platforms and sensitivity to firmware delays and artifacts.Source: Deployment · pp. 11–12

Trajectory Tracking

  • RMSE was 0.11 m on the 10 s figure-eight and 0.20 m on the 5.5 s figure-eight; on Crazyflie’s 5.5 s trajectory, RAPTOR reached 0.19 m versus 0.17 m for a dedicated policy.The reported RMSE ranges were 0.07–0.19 m and 0.17–0.29 m, respectively; tracking error was mostly in the x-y plane, and tested trajectories were not necessarily the maximum the policy could support.Source: Trajectory Tracking · pp. 12, 14, Figure 5 · p. 13

Outdoor Tests

  • In outdoor wind of about 7 m/s, gusting to 10 m/s, the policy showed no instability at ground speeds up to 10 m/s and over 15 m/s relative to the wind.With a 2.4 kg take-off weight, it could hover; with a 600 g payload, it could still track trajectories.Source: Outdoor Tests · p. 14

Disturbances

  • The policy recovered after a >90° tilt, adapted to added weight, controlled vehicles with swapped mixed-blade propellers, and generalized to 10× longer contexts than training sequences.It also remained stable when six quadrotors flew together amid turbulence; in practice, the authors flew until batteries were empty without noticing a limit on activation duration.Source: Disturbances · pp. 14, 16, Figure 6 · p. 15

Simulation

  • On Flightmare, whose default thrust-to-weight ratio is approximately 12 versus the training range’s upper limit of 5, the policy controlled the vehicle robustly and tracked agile trajectories despite substantial z-error.Source: Simulation · p. 16

Discussion

RAPTOR approximately matches platform-specific state-of-the-art control while adapting to diverse quadrotors without retraining through in-context learning. Its tiny policy can run on the smallest quadrotors, including platforms whose parameters have not been identified.

Discussion

  • RAPTOR approximately matches state-of-the-art end-to-end control while enabling zero-shot adaptation across platforms without retraining and using substantially less computation.The tiny policy can also be deployed on the smallest quadrotors, including platforms whose parameters have not yet been identified.Source: Discussion · pp. 16–17

Simulation-to-reality gap in flight controller firmware implementations

  • Firmware and infrastructure delays, along with state estimation, account for a large part of the simulation-to-reality gap; proposed remedies include delay randomization and direct IMU input.The authors suggest moving toward a fully end-to-end architecture by removing state estimation.Source: Discussion · p. 17

Over-reliance on linear velocity observations

  • The policy over-relies on linear velocity observations for emergent system identification, which can be delayed or disturbed; proposed remedies are more observation noise and accelerometer inputs.The cited real-world concerns include motion-capture delay and GPS drift.Source: Discussion · p. 17

Limited domain randomization range

  • RAPTOR’s broad training distribution still has out-of-distribution limits: Flightmare tracking is lower at a thrust-to-weight ratio of ≈12 than at the training limit of ≤5.The authors report excellent Flightmare tracking when its thrust-to-weight ratio is limited to 5, suggesting a wider domain-randomization range could help.Source: Discussion · p. 17

4. Trajectory tracking

Lookahead-free trajectory tracking limits RAPTOR’s performance, and the authors believe integrating lookahead into the architecture could improve it.

4. Trajectory tracking

  • Lookahead-free trajectory tracking poses a limit, and integrating lookahead into RAPTOR may improve performance.Source: §4 · p. 17

Materials and Methods

RAPTOR is formulated as a partially observed control problem with quadrotor dynamics parameters fixed during each episode. Its student uses observation-action history to infer hidden state and dynamics while learning to match RL-trained teachers across sampled quadrotors.

Materials and Methods

  • The BAPOMDP represents quadrotor state, motor-command actions, partial observations, and dynamics parameters that are sampled once and remain constant per episode.Observations omit motor states and external disturbances; dynamics parameters include mass, geometry, thrust and moment coefficients, inertia, and motor delays.Source: Materials and Methods · pp. 17–18
  • RL trains 1000 teacher policies for 1000 sampled quadrotor dynamics, and a mixture conditioned on dynamics parameters provides the teachers’ optimal-action targets for the student.The finite teacher set is treated as a Monte Carlo approximation to a unified teacher conditioned on the dynamics parameters.Source: Materials and Methods · pp. 18, 20
  • Observation-action history can reveal hidden dynamics, such as motor-command delays and nonlinear thrust curves, motivating a policy that depends on past observations and actions.For example, a one-step delay or different acceleration from commands of 1 and 2 can indicate motor timing or thrust-curve curvature.Source: Materials and Methods · p. 20
  • The student is trained to approximate the optimal-action distribution by minimizing KL divergence, using converged expert policies because the true distribution is intractable.The paper refers to supplementary materials for the derivation of the MSE training objective from maximum likelihood estimation.Source: Materials and Methods · p. 20

3. Meta-Imitation Learning

RAPTOR trains one recurrent student to adapt across quadrotors by distilling expert controllers trained on randomized dynamics. Its method factorizes physically plausible quadrotor sampling, parallelizes expert pre-training, and uses on-policy imitation for student learning.

Domain Randomization

  • Factoring the quadrotor dynamics distribution enables efficient ancestral sampling while maintaining physically plausible correlations among computed quantities.The distribution covers mass, geometry, inertia, thrust curves, torque coefficients, and motor delays; independent uniform samples are used for root quantities, avoiding MCMC.Source: Domain Randomization · p. 21

Training Methodology

  • The authors abandoned end-to-end recurrent meta-RL after observing no convergence and costly sequential training, and instead separated expert pre-training from student post-training.They chose this division because separate RL policies could be trained across sampled quadrotors and the probabilistic model implied dependencies among dynamics parameters.Source: Training Methodology · p. 21

Pre-Training

  • All 1,000 expert policies reached high quality without cherry-picked seeds or case-by-case modifications despite widely varying quadrotor dynamics.Experts train separately with fully observed states; their objectives include recovering from random states and tracking slow trajectories. Training changes include SAC, longer runs, reward adjustments, removing curriculum, and ground-truth motor RPM states.Source: Pre-Training · p. 22

Meta-Imitation Learning

  • The recurrent student learns to infer quadrotor dynamics from observation-action history while imitating teachers’ motor commands through on-policy rollouts.The authors call this Meta-Imitation Learning: each quadrotor is a separate MDP, and the student uses a GRU with hidden dimensionality 16. After a 10-epoch teacher-rollout warmup, student training uses on-policy data without teacher actions during rollouts or trajectories from past policies.Source: Meta-Imitation Learning · p. 23

Computational Aspects

  • Independent expert training can scale across processors or machines, with a 31-minute per-run ceiling and distillation about three orders of magnitude less computationally intensive than pre-training.Communication and synchronization are only needed after pre-training, during distillation.Source: Computational Aspects · p. 23

Supplementary Materials · Supplementary Materials for RAPTOR: A Foundation Policy for Quadrotor Control · This PDF file includes:

The supplementary materials identify Figure S1, Movies S1–S8, and Data S1, and provide captions for the movies and data.

Supplementary Materials

  • Figure S1 and Movies S1–S8 and Data S1 are listed among the supplementary materials.Source: Figure S1 · p. 29

This PDF file includes:

Other Supplementary Materials for this manuscript:

The supplementary materials listed are Movies S1 to S8 and Data S1.

Other Supplementary Materials for this manuscript:

Quadrotor Dynamics

This section defines the L2F simulator’s quadrotor dynamics setup, including planar-vehicle assumptions and conventions for motor commands.

Quadrotor Dynamics

  • The L2F simulator uses equations of motion to model quadrotor dynamics.Source: Quadrotor Dynamics · p. 31
  • The model considers planar quadrotors, with each rotor force direction set to [0, 0, 1]⊤.Source: Quadrotor Dynamics · p. 31
  • Motor positions follow the Crazyflie front-right, back-right, back-left, front-left order, and motor commands are remapped for other layouts.Torque directions follow CCW, CW, CCW, CW; CW corresponds to [0, 0, 1] in the body frame, which follows FLU conventions.Source: Quadrotor Dynamics · p. 31

Linear Velocity Feedback Delay Mitigation

The section describes an accelerometer-based filter to mitigate linear-velocity delay on non-EKF platforms and reports reduced z-axis oscillations. The filter removes gravity using the current orientation estimate and applies an exponentially decayed accelerometer integral.

  • For non-EKF platforms, the method uses a discrete IIR-style accelerometer integral with exponential decay after subtracting gravity via the current orientation estimate.The integral can be interpreted as convolving accelerometer data with an Infinite Impulse Response filter.Source: Linear Velocity Feedback Delay… · pp. 31–32
  • The mitigation substantially reduces z-axis oscillations on non-EKF-based platforms.Source: Linear Velocity Feedback Delay… · p. 32

Sampling Quadrotors

The method builds a physically plausible quadrotor distribution by sampling linked thrust, mass, size, geometry, and inertia parameters. It also randomizes motor and disturbance properties, while a post-hoc analysis identifies a mass-size sampling choice that better matches observed quadrotors than the intended distribution.

Sampling Quadrotors

  • Mass is sampled by uniformly drawing size scale and mapping it cubically into the target range, then scaling a baseline thrust curve to the sampled thrust-to-weight ratio.This avoids the bias toward large quadrotors that uniform mass sampling would create; the baseline curve is taken from the Crazyflie (39).Source: Sampling Quadrotors · p. 32
  • The mass-size ratio variation used a normal distribution by mistake, yielding ≈7.24 ± 0.66, closer to the empirical ≈7.66 ± 1.93 than the intended ±10% uniform reciprocal deviation.The authors advise future work to investigate direct normal sampling with a wider standard deviation closer to the empirical spread.Source: Sampling Quadrotors · p. 33
  • The mass-size ratio sets arm length and symmetric-X geometry, while a sampled torque-to-inertia ratio of 40–1200 determines x and y inertia from thrust-to-weight, mass, and arm length.The z inertia is calculated using the rule of (39).Source: Sampling Quadrotors · p. 33
  • Moment constant and motor delays are sampled independently, and disturbance-force standard deviation is set to 10% of surplus thrust, keeping sampled forces within 30% of surplus thrust with ≈99.7% probability.The independent sampling reflects the authors’ experience that moment constant and motor delays do not strongly correlate with other variables.Source: Sampling Quadrotors · pp. 33–34

Sampling Trajectories

The method mixes position control with random trajectory tracking and samples trajectories incrementally using a second-order Langevin process. It works for moderately agile tracking and generalizes to Lissajous trajectories, but leaves important trajectory types uncovered or unlikely.

Sampling Trajectories

  • For a recurrent policy, trajectory tracking errors alter the apparent quadrotor dynamics, unlike Markovian pre-training where trajectory tracking and position control appear identical.The source notes this equivalence absent aerial drag forces from linear velocity.Source: Sampling Trajectories · p. 34
  • Expert and student policies train on a mixture of null-trajectory position control with 50% probability and randomly sampled trajectory tracking with 50% probability.Position and linear velocity are offset by the target trajectory in observations and reward, without changing the policy structure or parameters.Source: Sampling Trajectories · p. 34
  • Second-order Langevin sampling covers smooth reference functions and can be performed incrementally during quadrotor simulation.The process corresponds to a Gaussian process with a kernel structure determined by its parameters.Source: Sampling Trajectories · p. 34
  • The approach supports moderately agile trajectory tracking and generalizes from second-order Langevin random walks to cyclical Lissajous trajectories.The authors note that step-function responses are not covered or that cyclical trajectories are exponentially unlikely under the sampling process.Source: Sampling Trajectories · pp. 34–35

Meta-Imitation Learning Objective

RAPTOR trains a recurrent student to imitate quadrotor-specific teachers by minimizing their action-distribution KL divergence. The recurrent policy performs amortized inference at test time, and Meta-Imitation Learning uses an MSE loss under stated assumptions.

Meta-Imitation Learning Objective

  • The recurrent policy performs variational inference amortized across cases at test time, enabling efficient inference on tiny microcontrollers at high frequencies.Inference is conducted by the recurrent neural network policy rather than separately for each case.Source: Meta-Imitation Learning Objective · p. 35
  • RNNs are selected for fixed per-step inference compute and context-window extrapolation beyond training length, unlike the cited CNN and attention alternatives.The passage characterizes RNN inference as O(1) in history length, versus CNN O(log(N)) and attention O(N) per step.Source: Meta-Imitation Learning Objective · p. 35
  • The foundation policy is a three-layer dense–GRU–dense network with 2084 parameters, zero-initialized recurrent state, and previous-output feedback.Its input includes sequential observations and actions from earlier steps.Source: Meta-Imitation Learning Objective · pp. 35–36
  • The student is trained to minimize KL divergence between its predicted optimal-action distribution and the current dynamics-specific teacher’s distribution.The paper describes the teacher as approximating the optimal action distribution.Source: Meta-Imitation Learning Objective · pp. 35–36
  • Under unit-standard-deviation and unbiased-teacher assumptions, the KL objective reduces to an MSE loss optimized by gradient descent.The paper identifies the MSE as identical to its derived Eq. S54.Source: Meta-Imitation Learning Objective · p. 37

Related papers

Loading 2509.11481v2…