Source-linked AI summary
Battery-Aware Reinforcement Learning for Aggressive Quadrotor Flight
Alejandro Sanchez Roncero, Olov Andersson, Petter Ogren
TL;DR
Battery discharge and motor load reduce available thrust during aggressive flight, while conservative command limits leave some performance unused and shared motors can saturate during sharp turns. The approach couples an identified load-transient battery model to rotor dynamics and firmware saturation, and gives a feedforward policy filtered voltage alongside increased command authority. On the Crazyflie Brushless, the approach nearly halves tracking RMSE on the harder operational circle and reduces 20-lap race time by 10.3% relative to stock-authority policies.
Problem
Battery discharge and motor load reduce available thrust during aggressive flight, while conservative command limits leave some performance unused and shared motors can saturate during sharp turns.
Source: §I · p. 1Method
The approach couples an identified load-transient battery model to rotor dynamics and firmware saturation, and gives a feedforward policy filtered voltage alongside increased command authority.
Source: §I · pp. 1–2, C. Load-Transient Battery Model · p. 3Results
On the Crazyflie Brushless, the approach nearly halves tracking RMSE on the harder operational circle and reduces 20-lap race time by 10.3% relative to stock-authority policies.
Source: §V · p. 8Takeaways & Limitations
On the Crazyflie Brushless, explicit voltage feedback can complement flight-controller compensation, with improved hardware error and race time over voltage-blind policies using the same increased authority.
Source: §V · p. 8
Abstract
from arXiv · showhide
Agile flight tasks such as drone racing and pursuit-evasion require strong acceleration and precise turns, but the available thrust changes as the battery discharges and voltage drops under load. Conservative command limits make this variation easier to tolerate, at the cost of unused performance. We investigate how learned controllers can use that additional thrust while retaining the flight controller's voltage compensation and rate control. Our training simulator couples an identified load-transient battery model to rotor dynamics and firmware saturation. The feedforward policy receives filtered voltage during both training and deployment. Controlled ablations distinguish the benefit of a larger thrust-command range from that of voltage information. On a 38 g Crazyflie Brushless, the resulting policy reduces circle tracking error by 49% relative to stock-authority RL at 3.84 m/s, while preserving easy-task precision. Mean 20-lap race time decreases from 106.22 s to 95.24 s. Compared with a voltage-blind policy with the same increased authority, hardware error and race time are lower by 15.3% and 4.5%, respectively. In simulation, replacing the policy's voltage input with a recording from a different battery condition worsens hard-circle tracking, with a smaller, voltage-dependent effect in racing. Together, these results show where a simple voltage input complements existing actuator compensation in aggressive learned flight.
I. INTRODUCTION
The paper addresses battery-dependent thrust and motor saturation in aggressive flight by combining an identified actuation model with a voltage-conditioned policy. It separates the effects of increased command authority and voltage input, and relates this design to prior battery modeling and learned-flight approaches.
I. INTRODUCTION
- Battery discharge and high motor load reduce available thrust, while shared motors can saturate during sharp turns and force the mixer to redistribute commands.A maneuver feasible early in flight may become harder later as battery voltage falls.Source: §I · p. 1
- The comparisons test fixed-speed circle tracking and let policies choose their own path and speed in racing, with Stock, High, and High-V separating command range from voltage input.High raises the default command limit without voltage input; High-V adds a 10 s filtered voltage input.Source: §I · pp. 1–2
- The policy combines a relaxed thrust-command limit with filtered voltage while retaining onboard compensation and collective-thrust/rate control.The authors separately compare additional command authority and voltage information.Source: §I · pp. 1–2
- The simulator models load-driven voltage changes, rotor response, firmware compensation, and saturation; its battery model is identified from measurements and validated against flight data.This provides an electromechanical basis for training the policy.Source: §I · pp. 1–2
- Hardware validation on the Crazyflie Brushless shows improved performance over both stock-authority and voltage-blind high-authority policies.The supplied introduction passage does not quantify these improvements.Source: §I · p. 2
II. RELATED WORK
- Prior battery-control work includes load-transient and coupled propulsion models, while this paper predicts rapid load changes from duty and rotor-speed history without a current sensor.The added load-onset term improves high-load prediction on held-out flights.Source: §II · p. 2
- Swift models battery effects during training, whereas its reported actor inputs exclude battery information; this study tests an explicit voltage input with onboard compensation retained.The cited Swift work is marked.Source: §II · p. 2
- Other adaptation approaches use disturbance estimates, motion-history inference, or recurrent rotor-speed estimates, while this paper studies a scalar voltage input to a feedforward actor.The actor retains the existing collective-thrust/rate controller, and critic privilege is evaluated separately from flight-time voltage input.Source: §II · p. 2
III. METHOD
The vehicle and rotor model combines rigid-body motion, measured rotor force and torque curves, and motor lag. Its motor-speed target is extended to depend on voltage.
A. Vehicle and Rotor Dynamics
- The model represents position, velocity, body-to-world rotation, body rates, and the speeds of four rotors, together with mass, inertia, and gravity.These quantities define the vehicle and rotor state used by the dynamics.Source: A. Vehicle and Rotor Dynamics · p. 2
- Propulsive force sums measured rotor-thrust fits along the body thrust axis, while rotor moments combine force-arm moments and reaction, inertia, and acceleration torques.The model also includes body-frame aerodynamic drag.Source: A. Vehicle and Rotor Dynamics · p. 2
- Rotor speed follows first-order motor lag, retaining the identified thrust and torque curves while replacing the voltage-independent target with a voltage-dependent one.The motor duty is bounded between 0 and 1.Source: A. Vehicle and Rotor Dynamics · p. 2
B. Voltage-Conditioned Actuation
Voltage-conditioned actuation links terminal voltage to steady-state rotor speed while retaining firmware compensation, rate control, and motor-command constraints. The higher command scale changes the force requests without changing the physical motor model.
B. Voltage-Conditioned Actuation
- Even at full duty, rotor speed depends on terminal voltage, which includes the drop under motor load.Fitted gain and exponent set the steady-state rotor-speed target.Source: B. Voltage-Conditioned Actuation · pp. 2–3
- The onboard controller converts collective-thrust and rate commands into motor requests, applies voltage compensation, and clips duty after common desaturation when requests exceed the limit.Compensation uses filtered supply voltage, while the motors respond to terminal voltage.Source: B. Voltage-Conditioned Actuation · p. 3
- Raising the per-motor force scale from 0.20 to 0.25 N unlocks more existing hardware capability, and High and High-V use the same interface.The physical motor model is unchanged; the scale affects collective and PID differential requests.Source: B. Voltage-Conditioned Actuation · p. 3
C. Load-Transient Battery Model
The battery model represents gradual depletion and temporary load-induced voltage sag using duty and rotor-speed histories, without measuring current. Filtered load states and fitted coefficients produce terminal voltage for the simulator.
C. Load-Transient Battery Model
- The model captures both gradual battery depletion and temporary voltage drops under rising motor load from duty and rotor speed, without a current measurement.Its internal states summarize load history rather than electrochemical state of charge.Source: C. Load-Transient Battery Model · p. 3
- Filtered states track duty load at fast and slow time scales, rotor-speed load, and squared duty load to represent voltage response and recovery.The state updates use time constants for the different inputs.Source: C. Load-Transient Battery Model · p. 3
- Fitted coefficients scale voltage drops from a reference, and an added load-onset term captures sharp dips without forcing a recovery overshoot.The term vanishes at steady load.Source: C. Load-Transient Battery Model · p. 4
- At reset, the model initializes to a rested-equivalent voltage; the optional seven-value simulator battery vector is available to the critic, while the actor receives only filtered voltage when enabled.The reset voltage is not an in-flight open-circuit measurement.Source: C. Load-Transient Battery Model · p. 4
D. Policy and Training Information
The policy operates in a partially observable setting where battery state can affect available force without being observed. Its actor combines flight and task inputs with optional filtered voltage, while training uses a critic to estimate future reward.
D. Policy and Training Information
- Battery state affects available force even when unobserved in the partially observable task formulation.The state includes vehicle, task, motor and battery variables, controller and observer histories, delays, and randomized parameters.Source: D. Policy and Training Information · p. 4
- The feedforward actor maps its current observation to an action and is trained with PPO to maximize expected discounted reward.Deployment uses the clipped mean of the policy's sampled actions.Source: D. Policy and Training Information · p. 4
- Racing inputs identify the active gate and its world-frame corners, while tracking inputs include future reference-position increments rather than reference velocity or acceleration.The reference increments use preview offsets; racing observes corners of the next two gates.Source: D. Policy and Training Information · p. 4
- The actor observation combines flight inputs, task inputs, and optional filtered voltage simulated in training and measured in flight.The voltage filter initializes from startup voltage; a constant-voltage control tests whether the extra input dimension alone explains a gain.Source: D. Policy and Training Information · p. 4
- The training-only critic estimates discounted future reward using either the actor's observations alone or additional battery state.The battery-privileged critic processes the actor observations and battery state with separate networks before predicting a scalar value.Source: D. Policy and Training Information · p. 4
E. Tasks and Rewards
The paper evaluates fixed-speed circle tracking and gate racing with task-specific rewards. Circle tracking rewards position and heading accuracy, while racing rewards progress and valid gate crossings subject to rate and course-deviation penalties.
E. Tasks and Rewards
- Circle tracking uses a counter-clockwise circle at prescribed speed and rewards accurate position and heading while penalizing large body rates and abrupt action changes.Ideal collective thrust balances gravity and centripetal acceleration before drag and differential-control losses.Source: E. Tasks and Rewards · p. 4
- The circle reward uses two position-error scales, heading error, body rates, and clipped action changes, with excessive tracking error, invalid state, or arena exit ending the episode.The position scales are mixed by λp; the w coefficients weight reward contributions.Source: E. Tasks and Rewards · p. 4
- Gate racing rewards progress toward the next gate and valid crossings in prescribed order while penalizing large body rates and course deviations.Missed gates, corridor violations, contact, invalid state, and arena exit incur a failure cost.Source: E. Tasks and Rewards · pp. 4–5
IV. EXPERIMENTS
The experiments first validate the battery model, then compare hardware performance and examine actor-voltage and training-only battery-information effects.
IV. EXPERIMENTS
- The experiments first validate the battery model.The section then proceeds to hardware performance comparisons.Source: §IV · p. 5
- The experiments compare hardware performance across policies.This follows battery-model validation.Source: §IV · p. 5
- The experiments examine the effects of actor voltage input and training-only battery information.These effects are examined after the hardware performance comparison.Source: §IV · p. 5
A. Platform and Learning Protocol
The protocol uses a Crazyflie Brushless platform and simulation-based policy training, then selects policies by held-out performance under safety criteria before hardware evaluation. Circle tracking and racing use specified task settings, and the protocol includes voltage-prediction and tracking evaluations.
A. Platform and Learning Protocol
- The 38 g Crazyflie Brushless is tracked by 100 Hz motion capture, with onboard logs recorded at up to 500 Hz during identification and 250 Hz in final flights.Isaac Lab on Isaac Sim runs vehicle dynamics and firmware control at 500 Hz, with a 50 Hz actor and 10 ms action latency.Source: A. Platform and Learning Protocol · p. 5
- Circle tracking tests a 1 m radius at 1.15 m height, with prescribed speeds of 3.36 and 3.84 m/s and 4.08 m/s as a stress test.Ideal collective demand rises from 0.57 to 0.73 N before drag and attitude corrections.Source: A. Platform and Learning Protocol · p. 5
- The protocol evaluates measured and predicted battery voltage on held-out flights from two excitation missions, with lower panels zooming into load increases to show the load-onset term's effect.Source: Figure 4 · p. 5
- The racing course has twelve upright 0.60 m virtual gates, trained with a 0.35 m crossing window and gate-center jitter of ±0.05 m.Training starts mix perturbed hover and recovery states at 0.5–2.5 m/s tangent speed.Source: A. Platform and Learning Protocol · p. 5
- Circle tracking compares hardware and nominal simulation at three speeds over 1.5–10 s after the entry ramp, with all 50 flights shown.The 4.08 m/s condition is the stress test; simulation lines show shaded standard deviation.Source: Figure 5 · p. 6
- Policies are selected on held-out simulation performance at reset voltages from 3.70 to 4.20 V before hardware evaluation, subject to failure and arena-exit rates below 1% at each voltage.Selection minimizes mean RMSE or flying-lap time at 3.70, 3.95, and 4.20 V; primary selection uses three seeds per task.Source: A. Platform and Learning Protocol · p. 6
B. Model Identification and Validation
The battery model improves voltage prediction under load, and hardware tests show that increased authority with voltage input improves hard-circle tracking and racing against both stock-authority and voltage-blind policies. The validation also identifies boundaries: simulation underestimates stress-speed tracking error, and High-V gains race time through speed rather than a shorter route.
B. Model Identification and Validation
- Motor response and the base battery model are identified from command steps and sustained loads, yielding α = 880.35 rad s−1 V−β, β = 0.802, and τm = 50 ms.Identification uses recorded duty, RPM, and terminal voltage without a current sensor.Source: B. Model Identification and Validation · p. 6
- In leave-one-flight-out validation, the load-onset term lowers voltage RMSE from 50.2 to 32.1 mV and high-load 95th-percentile absolute error from 161.9 to 83.6 mV.The base battery model was held fixed while fitting this term; model structure was selected during the campaign, so validation supports voltage prediction, not each term’s control benefit.Source: B. Model Identification and Validation · p. 6
C. Hardware Performance and Transfer
- At 4.08 m/s, High-V tracks best while High falls behind Stock; simulation underestimates error even over the matched 1.5–10 s window.A focused High replay found lower collective requests and more motor saturation in hardware; estimator errors, controller history, and maneuver-dependent moments remain possible contributors.Source: C. Hardware Performance and Transfer · p. 7
- All 27 evaluated races complete the requested laps without failure or crash, within a 4.8 × 4.0 m motion-capture area.The course uses short acceleration segments, tight turns, and height changes; High-V follows a similar, slightly longer route than High and gains time through higher mean speed.Source: C. Hardware Performance and Transfer · p. 7
- At 3.84 m/s, High-V reduces circle RMSE 49.3% versus Stock; over 20 laps, it cuts Stock time 10.3% and is 4.5% faster than High.High-V achieves 7.23 cm RMSE at 3.84 m/s; its mean per-flight peak speed is 6.95 m/s versus 6.08 m/s for Stock.Source: Table III · p. 7, C. Hardware Performance and Transfer · p. 7
- Stock has the lowest electrical replay error, while High-V has the closest agreement between simulated and measured lap times.For deployed checkpoints, nominal simulation predicts mean flying-lap time within approximately 0.10 s of hardware.Source: C. Hardware Performance and Transfer · pp. 7–8
D. Which Voltage Input Helps?
Voltage-filter ablations show that the input’s usefulness depends on its filtering and the task. The retained 10 s filter supplies slowly varying battery-condition information while onboard compensation handles faster voltage changes.
D. Which Voltage Input Helps?
- Short and slow voltage filters improve 3.84 m/s circle tracking at fixed high authority, while a constant 3.95 V input stays close to High.At the 4.08 m/s stress speed, training with the 54 ms filter is less reliable, with off-scale error marked in the figure.Source: D. Which Voltage Input Helps? · p. 8
- High-V retains a 10 s filter for slowly varying battery-condition information, leaving faster voltage changes to onboard compensation.The 54 ms and 5 s alternatives fail the stated replacement criteria: at least 3% improvement on both the 3.84 m/s circle and racing, no more than 3% regression at 3.36 m/s, and failure below 1%.Source: D. Which Voltage Input Helps? · p. 8
- With the battery initialized at 4.20 V, supplying the 3.70 V recording substantially worsens circle tracking; in racing, incorrect voltage increases lap time by 1.84% at 3.70 V but has approximately zero mean effect at 4.20 V.The policy and physical battery remain unchanged in these paired simulations; both runs evolve freely.Source: D. Which Voltage Input Helps? · p. 8
E. Does Battery Information Help the Critic?
Battery information improves critic return prediction in racing, while asymmetric training improves mean performance across high-authority variants but stock regresses. The Crazyflie results show gains from voltage-aware high authority, and future work targets motion-history adaptation and current and power prediction.
E. Does Battery Information Help the Critic?
- Battery information substantially improves the critic’s return prediction in racing; circle critics already explain most return variation, especially at 3.84 m/s.This leaves less room for improvement in circle prediction accuracy.Source: E. Does Battery Information Help the… · p. 8
- Asymmetric training improves mean performance for every high-authority variant, while Stock regresses.Battery information may stabilize policy updates by explaining return differences from changing thrust availability; for Stock, conservative commands and compensation may make it less useful, while added critic complexity could hinder optimization.Source: E. Does Battery Information Help the… · p. 8
V. CONCLUSIONS
- On the Crazyflie Brushless, the approach nearly halves tracking RMSE on the harder operational circle and reduces 20-lap race time by 10.3% relative to stock-authority policies.It also improves on voltage-blind policies with the same increased command range.Source: §V · p. 8
- Future work will combine voltage feedback with motion-history adaptation and extend the model to current and power prediction.The stated aim is to let policies balance aggressive flight with energy consumption.Source: §V · p. 8