PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning-Oriented Fault-Tolerant Control for UAVs with Time-Varying Fault Estimation

Moh Kamalul Wafi

arXiv:2610.30566v1eess.SYcs.SY

TL;DR

UAV actuator and sensor faults can degrade control and threaten flight safety, motivating fault-tolerant strategies. The paper combines PI fault estimation with a DDPG-based controller trained on yaw; simulations report observer robustness under specified fault conditions and accurate tracking across three fault scenarios.

  • Problem

    Actuator and sensor faults are difficult to estimate in UAVs with coupled nonlinear dynamics and multiple sensors and actuators, yet left uncompensated they can degrade performance or destabilize flight.

    Source: §1 · p. 1

  • Method

    The framework combines a PI observer for actuator and sensor faults with an active fault-tolerant architecture and a DDPG policy trained using only the yaw channel.

    Source: §1 · p. 2, §8 · p. 15

  • Results

    The PI observer has exponential error convergence for constant faults without disturbances and ultimate boundedness for slowly varying faults under bounded disturbances; simulations report accurate tracking across three fault scenarios.

    Source: §4 · p. 8, §8 · p. 15, §7 · p. 15

  • Takeaways & Limitations

    The framework combines model-based fault estimation with learning-based control to improve fault accommodation and tracking performance in UAV systems.

    Source: §8 · p. 15

    Experimental validation on the Quanser QDrone 2 is left for future work.

    Source: §7 · p. 15

Abstract

from arXiv · show

The increasing deployment of unmanned aerial vehicles (UAVs) in safety-critical applications has created a growing demand for control strategies that remain reliable in the presence of actuator and sensor faults. This paper presents a fault-tolerant control framework that combines constructive mathematical modeling, proportional--integral (PI) fault estimation, and learning-oriented control. A nonlinear quadcopter model is first developed using both the Newton--Euler and Euler--Lagrange formulations, from which a discrete-time linear model is derived for observer and controller design. A PI observer is then proposed to estimate actuator and sensor faults, with exponential estimation-error convergence established for constant faults in the absence of disturbances and ultimate boundedness established for slowly time-varying faults under bounded disturbances. The estimated fault signals are incorporated into an active fault-tolerant control architecture together with a Deep Deterministic Policy Gradient (DDPG) controller, which is trained using only the yaw channel and subsequently applied to improve closed-loop tracking performance. Numerical simulations illustrate the observer's estimation performance under constant and slowly time-varying faults, including bounded disturbances, and the tracking performance of the integrated controller under multiple actuator and sensor fault scenarios. The proposed framework combines model-based fault estimation with learning-based control to improve fault accommodation and tracking performance in UAV systems.

1 Introduction

UAVs face actuator and sensor faults, disturbances, and uncertain environments that can degrade control and threaten flight safety. The paper combines PI fault estimation with learning-based control, training DDPG on yaw and applying the learned policy across multiple states.

1 Introduction

  • UAVs increasingly serve civilian and military applications, but adverse weather, component faults, and external disturbances can compromise control performance and flight safety.Applications include surveillance, inspection, environmental monitoring, search and rescue, and autonomous transportation.Source: §1 · p. 1
  • Fault estimation is difficult in UAVs because coupled nonlinear dynamics and multiple sensors and actuators complicate reconstruction of actuator and sensor faults.Uncompensated faults may degrade performance, destabilize the system, or cause mission failure.Source: §1 · p. 1
  • Reliable UAV dynamics provide a foundation for designing fault-estimation and control algorithms, including classical and learning-based controllers.The introduction also situates control needs alongside resilient communication, navigation, and security research.Source: §1 · pp. 1–2
  • The proposed framework combines a PI observer with DDPG, training the agent on one output channel and applying its learned policy to improve control across multiple system states.The paper contrasts this with approaches requiring learning for every controlled variable.Source: §1 · p. 2

2 Dynamics of Quadcopter

The paper derives quadcopter dynamics using Newton–Euler mechanics and an alternative Euler–Lagrange formulation, then linearizes the inner-loop model about hover for estimation and controller design.

2.1 Mathematical Model of Quadcopter

  • The Newton–Euler model relates inertial and body frames, rotor forces, and rigid-body translational and rotational dynamics.The model defines position and attitude with ζ and η, and relates body angular velocity to Euler-angle rates through a kinematic transformation.Source: §2.1 · pp. 2, 4
  • Rotor thrust differences generate roll and pitch moments, while reaction-torque imbalance generates yaw; collective thrust drives vertical motion.The model assumes identical propeller–motor assemblies and a geometrically symmetric quadcopter.Source: §2.1 · p. 3

2.2 Alternative Lagrangian Derivation of the Equations of Motion

  • The Euler–Lagrange derivation uses generalized coordinates and kinetic and potential energies, yielding rotational dynamics equivalent to the Newton–Euler result.The translational model is augmented with aerodynamic drag, and rotor angular velocities are recovered from thrust and control torques.Source: §2.2 · pp. 5–6

2.3 Linearized System and Control Inputs

  • For control design, the twelve-state model is restricted to inner-loop dynamics and linearized about hover before discretization into a state-space model.The hybrid architecture uses PID actions for altitude, roll, and pitch, and a reinforcement-learning input for yaw.Source: §2.3 · pp. 6–7

3 Fault Modeling and Problem Formulation

The section models actuator and sensor faults, including slowly varying faults, and frames estimation and compensation as part of a tracking-preserving control design.

3 Fault Modeling and Problem Formulation

  • The framework explicitly models slowly time-varying faults, addressing gradual degradation that constant-fault assumptions may not describe well.The paper notes that constant-fault models may suit abrupt failures but not gradual degradation in aerial vehicles.Source: §3 · p. 7, §3.2 · p. 7

3.1 Actuator and Sensor Fault Modeling

  • The discrete-time model represents actuator and sensor faults alongside system states, control inputs, measured outputs, and unknown disturbances.Fault distribution matrices specify the channels in which actuator and sensor faults are represented.Source: §3.1 · p. 7
  • Actuator faults enter the state dynamics, while sensor faults corrupt measured outputs, so estimation must reconstruct both signals from measurements.The estimation objective also includes maintaining satisfactory closed-loop performance.Source: §3.1 · p. 7

3.2 Time-Varying Fault Model

  • The time-varying fault model treats changes between sampling instants as bounded, expressing the assumption that actuator and sensor faults evolve slowly.The constant-fault case is recovered when both fault-variation bounds are zero.Source: §3.2 · p. 7

3.3 Problem Formulation

  • The formulation targets fault estimation while preserving closed-loop tracking performance for systems satisfying the slow-variation assumption.The considered faults may arise in practical systems where constant-fault assumptions inadequately represent gradual degradation.Source: §3.2 · p. 7, §3.3 · p. 8
  • The design objectives are a PI observer for slowly varying faults, compensation using estimated faults, and a learning-oriented controller for tracking under bounded time-varying faults.Together these objectives specify the observer, compensation, and control components of the proposed problem formulation.Source: §3.3 · p. 8

4 PI Observer for Time-Varying Fault Estimation

The section develops a PI observer for actuator faults and extends it to combined actuator and sensor faults. Its guarantees distinguish exponential convergence for constant faults from bounded estimation error under bounded fault variation and disturbances.

4.1 PI Observer for Actuator Faults

  • The actuator-fault PI observer augments the state with the fault and estimates both using proportional and integral gains.The derivation omits disturbances initially; bounded disturbances can be included but generally introduce a nonzero error bound.Source: §4.1 · p. 8

4.2 Observability Condition

  • With (A, C) observable, a necessary-and-sufficient rank condition determines observability of the augmented pair, enabling selection of a Schur-stable observer gain.The condition must hold for the chosen fault channels.Source: §4.2 · p. 9

4.3 Estimation Error Bound for Time-Varying Faults

  • Bounded actuator-fault variation yields an input-to-state-stable estimation-error system with an ultimate error bound.The result assumes an observable augmented pair and a Schur-stable observer matrix.Source: §4.3 · pp. 9–10
  • If actuator-fault increments are O(T^2), the estimation error is ultimately bounded within an O(T^2) neighborhood.For constant actuator faults, the state and actuator-fault estimation errors converge exponentially to zero.Source: §4.3 · p. 10

4.4 Combined Actuator and Sensor Faults

  • For combined actuator and sensor faults, the PI observer gives exact asymptotic reconstruction for constant faults without disturbances and practical reconstruction under bounded disturbances and slow fault variation.With bounded disturbance and fault variation, the combined estimation error is ultimately bounded.Source: §4.4 · pp. 11–12

5 Fault-Tolerant Control

The architecture uses PI-observer fault estimates to compensate sensor measurements and actuator inputs online. Under observer convergence, fault effects are asymptotically removed; for slowly varying faults, residual effects remain bounded under the observer-stability conditions.

5 Fault-Tolerant Control

  • PI-observer fault estimates are reconstructed and compensated before faults propagate through the closed-loop system.The architecture integrates this compensation into active fault-tolerant control rather than redesigning the controller after a fault.Source: §5 · p. 12
  • Actuator compensation assumes matched additive faults, Ea = B, and incorporates the actuator-fault estimate into the compensated control input.That input is transmitted to the UAV actuators.Source: §5 · p. 12
  • When the observer converges, actuator and sensor fault effects are asymptotically removed; for slowly varying faults, residual fault effects remain bounded under the observer-stability conditions.The resulting tracking influence depends on the feedback controller’s robustness properties.Source: §5 · p. 12

6 Learning-Oriented Control

The learning-oriented controller adopts DDPG to address nonlinear uncertainties and improve tracking, with a deterministic policy mapping observations to continuous actions. Its policy is trained on yaw and then integrated with PI-observer fault compensation.

6 Learning-Oriented Control

  • DDPG is adopted to compensate for nonlinear uncertainties and improve closed-loop tracking performance.Its deterministic policy directly maps observed states to continuous control actions.Source: §6 · pp. 12–13
  • The DDPG training architecture uses an actor–critic design, with the critic approximating the action-value function used for policy gradients.The actor and critic parameters are updated by gradient ascent and the described actor update, respectively.Source: §6 · p. 13
  • The reinforcement-learning controller is designed for yaw y4 = ψ, while altitude, roll, and pitch use the nominal controller; the trained policy is integrated with PI-observer fault compensation.The agent is trained without actuator or sensor faults.Source: §6 · p. 13, Figure 3 · p. 14
  • The reward encourages the agent to keep yaw tracking error within a prescribed tolerance band during training.The observation-space bounds include the yaw state, tracking error, and its integral.Source: §6 · p. 13

7 Numerical Examples and Simulations

Numerical simulations evaluate PI-observer estimation and integrated fault-tolerant tracking across actuator and sensor fault scenarios. The controller maintains accurate trajectory tracking in all three considered scenarios, while the observer tracks slowly varying actuator faults despite bounded disturbances.

7 Numerical Examples and Simulations

  • The DDPG agent is trained for 100 s over 51 episodes using randomly generated reference trajectories and a yaw-state observation vector.The stated reward is 10 when |e| < 0.01 and 1 otherwise; training uses discount factor γ = 0.9 and mini-batch size 64.Source: §7 · p. 14
  • The DDPG controller achieves improved tracking accuracy and smoother responses than conventional PID for attitude states y2, y3, and y4 despite yaw-only training.Figure 4a illustrates actuator loss-of-effectiveness effects on UAV dynamics, while Figure 4b compares PID and DDPG tracking.Source: §7 · p. 14
  • The PI observer’s estimates are evaluated under actuator-only and combined actuator–sensor fault scenarios before being used to compensate corrupted inputs and measurements.The complete controller evaluation then uses those estimated faults.Source: §7 · p. 14
  • The integrated PI-observer-based FTC and hybrid PID–DDPG controller maintains accurate trajectory tracking under all three considered fault scenarios.The scenarios include simultaneous actuator faults, sequential actuator faults, and actuator faults followed by sensor faults.Source: §7 · pp. 14–15
  • The PI observer accurately tracks time-varying actuator faults despite bounded disturbances.The observer gain is obtained by pole placement with poles uniformly distributed in [0.2, 0.75].Source: §7 · p. 15
  • Experimental validation on the Quanser QDrone 2 is identified as future work.Source: §7 · p. 15

8 Conclusions

The paper combines PI fault estimation with reinforcement-learning-based control for UAVs and evaluates the integrated framework in numerical simulations. Its observer analysis gives convergence guarantees under specified fault and disturbance conditions, while broader multi-agent and communication-aware extensions remain future directions.

8 Conclusions

  • The proposed UAV framework combines a PI observer for actuator and sensor faults with a DDPG controller in an active fault-tolerant architecture.The quadcopter model is developed using Newton–Euler and Euler–Lagrange formulations.Source: §8 · p. 15
  • The observer analysis establishes exponential estimation-error convergence for constant faults without disturbances and ultimate boundedness for slowly time-varying faults under bounded disturbances.Source: §8 · p. 15
  • Numerical simulations illustrate PI-observer estimation under constant and slowly time-varying faults, including bounded disturbances, and integrated-controller tracking under actuator and sensor faults.Source: §8 · p. 15
  • Future directions include cooperative multi-agent UAV control, adaptive observers for rapidly time-varying faults, and communication-aware resilient control under network uncertainties and adversarial disturbances.Source: §8 · p. 15
Loading 2610.30566v1…