Source-linked AI summary
Dual Gradient Consensus for Adaptive Learning to Transient Out-of-Distribution Disturbances in Quadrotor Control
Mingtian Tan, Vasileios Tzoumas
TL;DR
Short out-domain residuals can corrupt online disturbance adaptation, while persistent domain changes still require learning. DGC uses two gradient memories and a confidence map to adjust the update step while retaining DAPL’s residual-learning pipeline. In numerical quadrotor simulations, it achieves the lowest mean position RMSE in all four cases.
Problem
Online adaptation must handle residual streams that mix persistent in-domain changes with short out-domain events, without letting the latter rewrite the learned disturbance model.
Source: §I · p. 1Method
DGC retains DAPL’s RFF residual model, offline BPTT policy, and least-squares loss, using long- and short-term gradient memories to classify regimes and adjust the online step length.
Source: §I · pp. 1–2, D. Dual Gradient Consensus · p. 3, E. Algorithm · p. 4Results
DGC achieves the lowest mean position RMSE in all four simulated cases; in Case 2, it remains at 0.006 ± 0.001 m versus 0.162 ± 0.044 m for DAPL.
Source: C. Numerical Simulations · p. 6, §VI · p. 7Takeaways & Limitations
In these numerical quadrotor simulations, DGC’s update limits the influence of short out-domain impulses while preserving adaptation to domain shifts.
Source: §VI · p. 7The evaluation is numerical only, and the paper provides no closed-loop stability argument for the gated step length.
Source: §V · p. 7
Abstract
from arXiv · showhide
Quadrotor trajectory tracking under unmodeled disturbances often relies on online residual adaptation, but short out-domain residuals can corrupt the learned disturbance model when constant-step updates treat every residual gradient uniformly. We propose Dual Gradient Consensus (DGC), an adaptive-learning method for mixed in-domain/out-domain disturbances that retains the random Fourier feature model, offline BPTT policy, and least-squares adaptation of a prior domain-adaptive residual policy (DAPL) while modifying the online update step size. DGC compares each residual gradient with long- and short-term gradient memories, classifies the current adaptation regime, and uses a confidence map to suppress unreliable updates while recovering adaptation after persistent domain shifts. We evaluate DGC in numerical quadrotor trajectory tracking under four residuals that mix an in-domain component with short out-domain events. Compared with $L_1$-augmented MPC, RFF-based MPC, constant-step DAPL, and a frozen-parameter baseline, DGC achieves the lowest mean position RMSE in all four cases.
I. INTRODUCTION
The paper addresses how to adapt to persistent domain changes without allowing short out-domain residuals to rewrite the learned disturbance model. It proposes DGC for mixed disturbances and evaluates it in numerical quadrotor trajectory-tracking simulations.
I. INTRODUCTION
- Persistent domain changes and brief transients both challenge trajectory tracking, but only persistent changes should shape the learned disturbance model.The residual stream can mix current-domain samples with short out-domain events, motivating adaptation that distinguishes the two.Source: §I · p. 1
- Prior approaches include robust control, adaptive estimators, learned controllers, online non-stochastic control, and predictive residual models; residual updates often use a uniform step.MPC (RFF) adapts random-feature coefficients and bandwidth, while DAPL applies constant-step least squares to the same pair.Source: §I · p. 1
- DGC retains DAPL’s RFF residual model, offline BPTT policy, and least-squares loss while changing the online step length.It compares each gradient with long- and short-term memories to classify adaptation regimes and modulate the update.Source: §I · pp. 1–2
- DGC achieves the lowest mean position RMSE across all four simulated disturbance cases, with its largest advantage under repeated impulses.The comparison includes MPC (L1-E), MPC (RFF), DAPL, and DAPL without σ and α updates.Source: §I · p. 2
II. PROBLEM FORMULATION
- The target problem is to track a reference while adapting only the in-domain disturbance estimate, although observed residuals mix in-domain effects with out-domain events.In-domain effects may be constant or periodic; out-domain effects are short-lived transients that should not be written into the learner.Source: §II · p. 2
III. ALGORITHM
The algorithm builds on DAPL’s residual-learning pipeline and replaces its constant online step size with Dual Gradient Consensus. Its residual model uses a random Fourier feature approximation of a function in an RKHS subspace.
III. ALGORITHM
- The algorithm recap identifies DAPL’s RFF residual, offline BPTT policy, and online least-squares kernel updates as the retained pipeline, with DGC replacing the constant online step size.The section also identifies the online procedures as DAPL (Algorithm 1) and DGC (Algorithm 2).Source: §III · p. 2
A. Function Approximation via Random Fourier Features
- The residual function is assumed to lie in an RKHS subspace and is approximated by a Monte Carlo truncation using random Fourier features.The feature map uses learned coefficients for sampled feature parameters.Source: A. Function Approximation via Random… · p. 2
- The RFF construction uses a Gaussian kernel bandwidth, Gaussian-sampled frequencies, and phases drawn uniformly on [0, 2π].The passage specifies the feature construction and sampling distributions used by the residual model.Source: A. Function Approximation via Random… · p. 2
B. Policy Optimization with Analytical Gradients
The policy is trained offline with BPTT, while online least-squares adaptation uses a filtered residual to form the learning signal. DGC retains that residual pipeline but replaces the constant update step with a consensus-based step length.
B. Policy Optimization with Analytical Gradients
- The policy is trained offline by BPTT on a differentiable closed-loop rollout, with its parameters updated using an analytic gradient.The supplied passage identifies the N-step objective and analytic-gradient update, but does not provide their full expressions.Source: B. Policy Optimization with Analytical… · p. 2
C. Online Least-Squares Estimation
- When the observed residual mixes in-domain and out-domain components, the raw residual is not a reliable supervised target, so the learning residual is low-pass filtered.The filter uses the short-memory timescale; the plant still uses the unfiltered residual prediction.Source: C. Online Least-Squares Estimation · pp. 2–3
- DGC’s overview shows the residual pipeline forming a packed gradient from the RFF predictor, a low-pass residual, and the least-squares loss.Long- and short-term gradient memories classify the gradient, and a confidence map sets the update step length.Source: Figure 1 · p. 3
- DGC keeps the least-squares residual pipeline and gradients but replaces the constant step that would treat every sample, including transients, equally.The constant-step update can remain unsafe when a gradient is still aligned with an unfinished transient.Source: C. Online Least-Squares Estimation · p. 3
D. Dual Gradient Consensus
Dual Gradient Consensus replaces the constant step in the unchanged residual pipeline with a regime-dependent update based on long- and short-term gradient agreement. It preserves the current-domain reference through transients and hands off to a new reference after a completed domain shift.
D. Dual Gradient Consensus
- DGC retains the residual pipeline and replaces only the step length after classifying the adaptation regime.The constant-step update treats every gradient equally, including gradients from transients and abrupt out-domain disturbances that may not be reliable descent directions.Source: D. Dual Gradient Consensus · p. 3
- The two-bit classes use positive cosine agreement with the long- and short-term gradient references, with both memories assumed nonempty in the definition.When the long-term memory is empty, both consensus and local multipliers are set to 1.Source: D. Dual Gradient Consensus · pp. 3–4
- DGC classifies gradients by agreement with long- and short-term memories, keeping the long-term reference in three regimes and handing it off to the short-term memory only after a completed shift.The four cases distinguish in-domain gradients, disagreement with both memories, a transient that pulls only the short-term memory, and a completed shift.Source: D. Dual Gradient Consensus · p. 3
- For a slowly varying wind or payload, both memories agree and DGC updates; for a short spike or collision, neither agrees and parameters are not replaced.If only the short-term memory is pulled by a transient, DGC keeps the long-term reference and still takes a confidence-scaled step.Source: D. Dual Gradient Consensus · p. 4
- The confidence map converts agreement with the working point into a step-size multiplier in [0, 1], shrinking updates on weak consensus and restoring base rates when the working point is trustworthy.A decrease is accepted only when raw prediction error is large enough; applied rates average the last W held rates.Source: D. Dual Gradient Consensus · pp. 3–4
E. Algorithm
The procedure trains the policy offline by BPTT, then adapts residual parameters online using DGC in place of DAPL’s constant step. Online updates use gradient-memory agreement, regime-specific reference handling, and confidence-scaled rates.
E. Algorithm
- Offline, BPTT trains the policy, which is held fixed during online operation; online, DGC replaces DAPL’s constant update step for the residual gradient.The online phase applies the policy, estimates the disturbance, forms the residual gradient, and passes it to DGC for parameter updates.Source: E. Algorithm · p. 4
- DGC forms mean references from pre-update gradient memories, applies the four-regime rule, and hands off the long-term memory only when the short-term reference indicates a completed shift.It then converts agreement with the possibly updated long-term reference into step sizes, conditionally holds decreases based on raw residual error, and averages applied rates over window W.Source: E. Algorithm · pp. 4–5
- After projected gradient descent updates the parameters on the filtered loss, the current gradient is appended to both FIFO memories.The algorithm outputs updated residual parameters and the two memories.Source: E. Algorithm · pp. 4–5
- The algorithm’s stated inputs include the gradient, residual parameters, memories, base rates, confidence-map settings, memory lengths, and error threshold.The output is the next residual-parameter pair together with the updated memories.Source: E. Algorithm · p. 5
IV. EXPERIMENTS
The experiments section introduces the evaluation sequence: training and algorithm parameters, benchmark controllers, then simulations.
IV. EXPERIMENTS
- The evaluation is organized into training and parameter setup, benchmark-controller descriptions, and simulations.These are presented in Sections IV-A, IV-B, and IV-C, respectively.Source: §IV · p. 5
A. Policy Training and Algorithm Parameters
The policy is a three-layer network trained offline by BPTT on a lemniscate tracking task. Its observation combines vehicle state, prior commands, future references, and kernel parameters.
A. Policy Training and Algorithm Parameters
- The policy is a three-layer multilayer perceptron with hidden width 512, and its observation has dimension 187.The observation stacks body-frame velocity, quaternion attitude, previous thrust and angular-velocity command, ten future reference points over a 1.0 s horizon, and kernel parameters α and σ.Source: A. Policy Training and Algorithm… · p. 5
- Offline BPTT training uses 500 parallel environments for 300 epochs on lemniscate tracking, with episodes capped at 250 steps of 0.02 s.Training uses 25 Gaussian random Fourier features and finishes in 50 s on an RTX 4090.Source: A. Policy Training and Algorithm… · p. 5
- Online, α and σ are initialized to zero and 0.5, respectively, and both use base rates of 0.1 under DGC.The offline policy step size starts at 0.001 and follows cosine decay; α is sampled from a Gaussian and σ uniformly from [0.001, 1] during training.Source: A. Policy Training and Algorithm… · p. 5
B. Benchmark Algorithms
The benchmarks contrast MPC-based residual compensation, constant-step DAPL, and a frozen-parameter baseline. DGC uses DAPL’s policy, features, and loss, changing only how the online update step is selected.
B. Benchmark Algorithms
- MPC (L1-E) augments the nominal quadrotor model with an L1 state predictor.Source: B. Benchmark Algorithms · p. 5
- MPC (RFF) uses random Fourier features to parameterize the residual, injects it into sampling-based MPC, and updates α and σ online.Source: B. Benchmark Algorithms · p. 5
- DAPL applies a fixed step to both α̂ and σ̂ using a filtered least-squares loss; its frozen-parameter variant never replaces them after initialization.Source: B. Benchmark Algorithms · p. 5
- DGC retains DAPL’s policy, features, and loss, while classifying the residual regime to select the update step.The confidence map φ selects the step length for the (α̂, σ̂) update.Source: B. Benchmark Algorithms · p. 5
C. Numerical Simulations
Numerical 10-second quadrotor rollouts combine in-domain residuals with impulsive out-domain events across four disturbance profiles. DGC has the lowest mean position RMSE in every case, while its update rates fall during transient prediction-error spikes.
C. Numerical Simulations
- The four 10-second rollouts combine in-domain residuals with out-domain impulses: a step shift, repeated impulses on a stationary domain, a periodic shift, and a ramp shift.The applied acceleration residual is identical on all three axes; Figure 2 plots one coordinate.Source: C. Numerical Simulations · p. 5
- DGC has the lowest mean position RMSE among five controllers in every case.Figure 2’s position-error row is consistent with the Table III comparison.Source: C. Numerical Simulations · p. 6, Figure 2 · p. 6
- In Case 1, the pulse at t = 2 s lowers DGC’s step length; after the hold, its parameters follow the new level and prediction error returns toward zero.MPC (RFF) and MPC (L1-E) oscillate after the level change, with RMSEs of 5.660±1.508 m and 1.921±0.524 m, respectively.Source: C. Numerical Simulations · p. 6
- In Case 2, DGC remains at 0.006 ± 0.001 m while DAPL reaches 0.162 ± 0.044 m; both MPC variants also stay accurate at 0.017±0.002 m and 0.022±0.002 m.DGC lowers its update rate on shaded impulse intervals, while DAPL writes the spikes into its parameters.Source: C. Numerical Simulations · p. 6
- In Case 3, DGC has lower RMSE than DAPL (0.145 ± 0.004 m vs. 0.255 ± 0.056 m) and lower peak position error than the MPC baselines.The update rate dips on the mid-horizon pulse as prediction error peaks.Source: C. Numerical Simulations · p. 6
- In Case 4, DGC’s RMSE is 0.068 ± 0.010 m; DAPL (w/o σ, α update) reaches 0.531±0.001 m, while MPC (RFF) degrades on some rollouts to 0.565 ± 0.496 m.DGC’s rates drop on the pulse and recover on the triangle-shaped disturbances.Source: C. Numerical Simulations · p. 6
V. LIMITATIONS
DGC achieves the lowest mean position RMSE across all four simulated disturbance cases, with its largest advantage under repeated spikes. The evaluation is numerical only, and the paper provides no closed-loop stability argument for the gated step length.
V. LIMITATIONS
- The evaluation is numerical only, and the paper provides no closed-loop stability argument for the gated step length.These boundaries limit the evidence to simulation performance rather than a demonstrated stability guarantee.Source: §V · p. 7
- Broadband transients are handled through consensus of the two gradient memories, while the residual is already low-pass filtered.The residual prediction error acts only as a scalar hold on the DGC step size, not as a signal for extracting or rejecting high-frequency content.Source: §V · pp. 6–7
VI. CONCLUSION
- DGC attains the lowest mean position RMSE in all four cases, with the largest difference when constant-rate DAPL absorbs repeated spikes.Table III reports position RMSE in meters as mean ± standard deviation over ten rollouts; the supplied passages give no cell values.Source: §VI · p. 7, Table III · p. 7
APPENDIX A BENCHMARK ALGORITHM IMPLEMENTATION
The appendix specifies the implementation settings for the MPC baselines, constant-step DAPL and frozen-parameter baseline, and DGC. The RFF MPC changes the residual channel relative to L1-E, while DGC retains DAPL’s policy, features, and loss.
A. MPC (L1-E)
- MPC (L1-E) uses a 40-step horizon, scores 1,024 perturbed command sequences per control instant, and feeds its L1 residual estimate into MPC.The horizon is 0.8 s at Δt = 0.02 s; the forward model includes first-order thrust and rate lag.Source: A. MPC (L1-E) · p. 7
B. MPC (RFF)
- MPC (RFF) keeps the L1-E sampler, horizon, costs, and limits, changing only the residual channel to an RFF learner.Its 25-feature model uses an 11-vector input, and both α and σ are updated at every control step with rate 0.1.Source: B. MPC (RFF) · p. 7
C. DAPL and DAPL (w/o σ, α update)
- DAPL takes a projected online step of size 0.1 on (α̂, σ̂), while its frozen baseline holds both parameters at their initial values.Both use the BPTT policy; the DAPL online run starts from α̂ = 0 and σ̂ = 0.5.Source: C. DAPL and DAPL (w/o σ, α update) · p. 7
D. Ours
- DGC uses DAPL’s policy, features, and loss, with specified gradient-memory windows, confidence settings, and step lengths.The settings include N_L = 20, N_S = 5, τ = 0.5, k = 10, ε_min = 0.2, W = 10, and η_α = η_σ = 0.1.Source: D. Ours · p. 7