Source-linked AI summary
Adaptive Aerial Loco-Manipulation Policy Learning via LLM-Assisted Proactive Task Generation
Jingyu Ru, Xiangyuan Xie, Lixuan Jin, Jian An, Xingming Zhang, Binghao Li, Siyu Wang, Xiang Zhou, Lei Yuan, Hongli Xu, Yang Yu
TL;DR
Aerial arm motion changes system dynamics while the resulting response is only partially observed, and fixed task sampling can leave tracking difficulties underexplored. ALMA trains an adaptive contextual flight policy with action-conditioned prediction and LLM-guided arm-motion task generation. It reduces simulation position RMSE by up to 24.7% relative to the strongest baseline and transfers zero-shot to reduce real-world hover position RMSE by 79.6%.
Problem
Aerial arm motion alters system response that is only partially observed, while control rewards lack a direct target for useful history information and fixed motion sampling can leave tracking difficulties underexplored.
Source: §1 · p. 1Method
ALMA trains a contextual flight policy to predict future measurement representations from recent history and actions, while an LLM uses episode records to propose arm-motion tasks targeting poor tracking.
Source: §1 · p. 2Results
24.7% lower position RMSE than the strongest baseline under periodic arm commands in simulation, and 79.6% lower hover position RMSE than the real-world baseline during periodic arm motion after zero-shot transfer.
Source: §1 · p. 2Takeaways & Limitations
A single contextual flight policy transfers zero-shot across fixed arm configurations and continuous arm motion in the reported experiments.
Source: §5.5 · p. 9
Abstract
from arXiv · showhide
Loco-manipulation extends the workspace of conventional manipulation by combining mobility with physical interaction, while aerial loco-manipulation further expands this capability into three-dimensional space, enabling robots to reach environments inaccessible to ground-based systems. Moving to an aerial platform, however, gives rise to a demanding control problem, since the vehicle must track trajectories accurately in three dimensions while the motion of the arm and its interaction with the environment continuously shift the center of mass, alter the inertia, and impose force and moment disturbances on the body. Conventional controllers address these effects through explicit models and estimation, whereas learning-based controllers infer dynamics from experience, yet control rewards provide no direct target for the response information retained from history, and fixed task sampling can leave persistent tracking difficulties insufficiently explored. We present ALMA, an aerial loco-manipulation framework for adaptive flight control under external arm commands. ALMA employs a contextual flight policy that adapts from recent measurements and actions, using action-conditioned prediction of future measurement representations to train its context to retain response information relevant to flight control. To expand the experience available for this adaptation, an evolutionary process uses a large language model (LLM) to analyze observed control difficulties and propose new structures of arm motion for subsequent training. In simulation, ALMA reduces position root-mean-square error (RMSE) by up to 24.7% relative to the strongest baseline, while the same policy transfers zero-shot to the aerial manipulator and reduces hover position RMSE by 79.6% during periodic arm motion relative to the real-world baseline.
1 Introduction
Aerial arm motion changes system dynamics and creates partially observed responses that the flight controller must track through. ALMA trains a contextual flight policy with action-conditioned predictive supervision and LLM-guided task generation, then evaluates it in simulation and on an aerial manipulator.
1 Introduction
- Arm motion shifts mass distribution and inertia, while joint accelerations and contact impose reaction forces and moments on the aerial base.Available measurements only partially reveal the resulting system response.Source: §1 · p. 1
- Prior learning objectives organize context by task identity or train residual predictors, whereas action-conditioned future-measurement prediction targets response distinctions relevant to corrective actions.Fixed motion-pattern sampling can leave persistent tracking difficulties insufficiently explored.Source: §1 · pp. 1–2
- ALMA trains its contextual flight policy to predict future measurement representations from recent history and subsequent actions, supervising information retained for control.An LLM analyzes episode records to identify poor-tracking motion patterns and propose new arm-motion structures for later training.Source: §1 · p. 2
- 24.7% lower position RMSE than the strongest baseline under periodic arm commands, with zero-shot transfer reducing hover position RMSE by 79.6% against the real-world baseline.In simulation, position RMSE was also 24.5% lower under static arm commands.Source: §1 · p. 2
2 Related Work
Prior aerial-manipulation work learns payload context or dynamics corrections, while predictive-state methods offer a principle for training history representations. ALMA also draws on feedback-driven task-generation methods to guide external arm-motion proposals.
2.1 Aerial Manipulation
- Predictive-state methods supervise decision-making representations through future-observation prediction, a principle ALMA applies by predicting future measurement representations conditioned on recorded actions.Related aerial-manipulation methods instead organize payload-dependent context or estimate corrections to nominal dynamics.Source: §2.1 · p. 2
2.2 LLM-Assisted Task Generation
- Prior methods use language-model proposals or policy feedback to refine tasks; ALMA uses the current flight policy’s execution records to guide generated external arm motions.This adapts training experience as policy performance changes.Source: §2.2 · p. 2
- Figure 1 depicts predictive supervision and LLM-assisted task generation training the contextual flight policy, with only that policy retained for deployment.Execution records guide the arm-motion proposals during training.Source: Figure 1 · p. 3
3 Background
The problem is trajectory tracking under external arm commands when system responses are only partially observed. The formulation treats each reference-and-arm-command process as a task and optimizes a history-dependent policy across a performance-updated task distribution.
3 Background
- Arm motion and uncertain physical conditions produce partially observed responses, and the objective is one flight policy that adapts its actions to those responses.Arm commands are outside the flight policy’s action space, and access to their future values is not assumed.Source: §3 · p. 3
- Each admissible task specifies a flight reference and arm-command process, inducing a POMDP in which policy actions depend on observation-action history.Observations provide reference information and available measurements, while the state includes physical and hidden process variables.Source: §3 · p. 3
- Policy learning maximizes expected return over tasks sampled from a stage-specific distribution, which can be updated according to policy performance.Training begins from the base task distribution.Source: §3 · p. 3
4 Method
ALMA combines a contextual flight policy with predictive supervision and LLM-assisted task generation to adapt flight control under external arm commands. Training uses randomized physical conditions and feedback to adjust exposure to generated arm motions.
4.1 Contextual Flight Policy
- The policy selects individual rotor commands from reference, measurement, and recurrent-context features, allowing observed responses to inform corrections without a separate nominal allocator.Its GRU context summarizes recent actions and measured responses; individual rotor commands account for arm-induced changes to the system response.Source: §4.1 · p. 4
4.2 Predictive Context Learning
- Predictive supervision trains the shared context to predict future measurement representations from recent context and recorded rotor commands, complementing control-reward learning.The prediction objective favors retaining history distinctions relevant to future responses, while the control objective trains how that representation selects rotor commands.Source: §4.2 · pp. 4–5
4.3 Proactive Task Generation
- LLM-assisted task generation proposes additional arm motions from execution records, evaluates candidate variants for tracking difficulty, and retains selected motions for training.Tasks vary arm commands while retaining the tracking objective; lower mean tracking fractions receive priority, with position and yaw RMSE used to resolve ties.Source: §4.3 · pp. 5–6
4.4 Training Procedure
- Training combines randomized vehicle, arm, and payload conditions with adaptive sampling of generated tasks, reducing their probability when base-task performance deteriorates or crashes persist.Generated-task probability starts at 0.05, is capped at 0.20, and is zero when the retained task population is empty.Source: §4.4 · p. 6
5 Experiments
Experiments compare ALMA with flight-control baselines, test predictive-context and task-generation variants, and evaluate zero-shot transfer from simulation to real-world aerial manipulation.
5.1 Experimental Setup
- The benchmark tests five flight references under identical arm commands, scores position and yaw RMSE and success rate, and evaluates the same fixed-parameter policy after simulation training.The aerial platform is a 2.39 kg quadrotor with a 4-DoF arm and a maximum reach of approximately 0.62 m.Source: §5.1 · p. 6, Figure 2 · p. 7
5.2 Flight Tracking
- Across static and periodic arm commands, ALMA achieves the lowest mean position RMSE, reducing error versus NMPC, the strongest position-tracking baseline, by 24.5% and 24.7%, respectively; PID–INDI achieves the lowest yaw RMSE in both settings.Source: §5.2 · p. 7
5.3 Predictive Context Learning
- PRED-5 reduces position and yaw RMSE by 11.0% and 25.0% versus GRU ONLY; it has the lowest position RMSE, although PRED-1 has the lowest yaw RMSE.PRED-1's mean yaw RMSE is 0.011° lower than PRED-5's, but K = 5 is selected for position tracking; GRU ONLY also has lower yaw RMSE than FEEDFORWARD with similar position means.Source: Table 2 · p. 8, §5.3 · p. 8
- PRED-10 has higher mean position and yaw RMSE than PRED-5, and doubling the horizon halves the combined weight of the first five prediction targets.More distant targets may depend more strongly on future arm commands that are absent from the predictor input, potentially reducing emphasis on immediate predictable responses.Source: §5.3 · p. 8
5.4 Proactive Task Generation
- ALMA improves all six mean RMSE metrics and reduces OOD yaw RMSE by 22.8% versus RANDOM EVOLUTION; its 12.3% reserved-family yaw reduction contrasts with that method's 15.6% increase in OOD yaw error.Both methods use the same candidate search and selection procedure, while parameter search cannot introduce new joint coordination patterns or event sequences.Source: Table 3 · p. 8, §5.4 · pp. 8–9
5.5 Real-World Experiments
- 3.17 cm versus 15.55 cm: ALMA reduces hover position RMSE by 79.6% versus PID during 8 s periodic arm motion and transfers zero-shot to hardware.Trajectory experiments also report lower position RMSE on Figure-eight and ICRA references with the arm home and fully extended forward.Source: Figure 4 · p. 9, §5.5 · p. 9, Figure 3 · p. 9
6 Conclusions
ALMA combines predictive context supervision with LLM-assisted task generation, and the conclusion reports simulation and zero-shot real-world tracking gains alongside directions for future work.
6 Conclusions
- ALMA combines action-conditioned predictive supervision of policy context with LLM-assisted task generation guided by execution feedback.The analysis relates historical response information to actuator-dependent corrections.Source: §6 · p. 9
- Ablations support predictive supervision's tracking benefit and task generation's OOD generalization benefit.Source: §6 · p. 9
- 79.6% reduction in hover position RMSE relative to PID during periodic arm motion; simulation reports the lowest mean position RMSE, up to 24.7% below NMPC.The real-world result is from zero-shot deployment.Source: §6 · pp. 9–10
- Future work targets improved yaw tracking and contact-aware control with force feedback and contact constraints.Source: §6 · p. 10