Source-linked AI summary
Safety-Critical Bilateral Teleoperation for Omnidirectional Aerial Manipulation Using Force-Sensorless Haptic Feedback
Yubin Kim, Jinwoo Lee, Yongjun You, H. Jin Kim, Jeonghyun Byun
TL;DR
The paper addresses bilateral teleoperation of omnidirectional aerial manipulators without onboard force/torque sensors, while accounting for unreliable model-based estimates and overlooked thrust limits. It introduces a two-stage hierarchical safety filter that uses tracking-error and thrust constraints to adjust commands and convert those adjustments into haptic cues. Experiments show reduced unintended free-flight feedback, directionally consistent interaction feedback, diverse manipulation-task operation, and improved stability through saturation filtering.
Problem
Existing approaches rely on force/torque sensors or model-dependent wrench estimates, while thrust limits are often overlooked despite their importance for safe aerial manipulation.
Method
A two-stage hierarchical safety filter adjusts AM commands using tracking-error and motor-thrust constraints, mapping the adjustments into haptic feedback without onboard force/torque sensors.
Results
Hardware experiments show suppressed unintended free-flight feedback, responsive directionally consistent interaction feedback, multiple manipulation tasks, and saturation-filter mitigation of motor saturation and destabilization.
Takeaways & Limitations
The framework provides haptic cues for physical interaction and impending motor saturation while supporting safer, more stable omnidirectional aerial manipulation.
Abstract
from arXiv · showhide
This paper presents a safety-critical bilateral teleoperation framework for omnidirectional aerial manipulators that integrates visual and force-sensorless haptic wrench feedback. Unlike existing approaches that either rely on onboard force/torque sensors or use model-dependent wrench estimates, which may become unreliable under model uncertainties or induce unintended feedback during free-flight, our method implements a hierarchical safety filter based on control barrier functions to avoid such limitations. The safety filter, being the key contribution, explicitly accounts for tracking errors arising from physical interaction between the aerial manipulator and its surroundings while enforcing thrust limits, a factor overlooked despite its critical importance for flight safety. This safety filter adjusts the command from the operator to ensure safe and stable aerial manipulation and avoid motor saturation. The adjustment made by the filter is mapped to haptic feedback, which is intuitive to the operator and conveys information on physical interaction and impending motor saturation. By actual experiments with a hexarotor-based omnidirectional aerial manipulator, we demonstrate that the proposed method avoids haptic feedback during free-flight, provides directionally consistent feedback under physical interaction, and can be operated for diverse manipulative tasks. Moreover, an ablation study further shows that the saturation filter improves interaction stability by explicitly preventing motor saturation and informing the operator of corrective actions.
I. INTRODUCTION
The paper targets bilateral teleoperation for omnidirectional aerial manipulators when physical interaction demands informative haptic feedback without force/torque sensors. It combines sensorless haptic cues with thrust-aware safety filtering to support safer, more stable manipulation.
- Motivation: Bilateral teleoperation remains necessary for human adaptability in previously unknown environments, despite progress in autonomous aerial-manipulator control.
- Motivation: Model-dependent force/torque-sensorless wrench estimates may become unreliable under model uncertainties, while interaction feedback must provide clear, reliable cues.
- Motivation: Thrust-limit compliance is crucial for safe aerial-manipulator operation, and impending motor saturation should be communicated to the operator.
- Contributions: The proposed framework uses a hierarchical safety filter to adjust commanded pose using interaction-induced tracking errors and thrust limits, then maps adjustments into haptic feedback.
- Contributions: The paper claims a first framework combining motor thrust limits, force/torque-sensorless haptic feedback, and fully omnidirectional aerial manipulation.
- Experiments: Hardware demonstrations evaluate the framework during static and dynamic physical interaction.
C. Outline
The framework pairs a robotic-manipulator master with an omnidirectional aerial-manipulator slave. Master commands pass through hierarchical interaction and saturation filters before desired states are tracked by the aerial-manipulator controller.
- B. Outline: The architecture consists of a robotic manipulator serving as master and an omnidirectional aerial manipulator with tiltable rotor arms serving as slave.
- B. Outline: The master admittance controller combines estimated external wrench and safety-filter feedback wrench, then maps pose deviation from neutral into the AM twist command.
- Safety Filter: The interaction filter modifies the command twist to satisfy tracking-error constraints and generate feedback reflecting environmental interaction.
- Safety Filter: The saturation filter further modifies the interaction-filter output to ensure motor thrust limits are not violated.
- AM Control: The filtered command is integrated into desired pose, twist, and spatial acceleration, which the gRITE controller and geometric allocation convert into thrust magnitudes and rotor tilt angles.
III. MASTER DEVICE
The master device is modeled as a revolute-joint open chain whose dynamics support force-sensorless external-wrench estimation and admittance-based end-effector control.
- III. MASTER DEVICE: The master device is modeled as an open chain of rigid bodies connected by nm revolute joints.
- III. MASTER DEVICE: Its Euler–Lagrange dynamics include inertia, Coriolis and centrifugal effects, gravity, command joint torques, and external wrench contributions.
- III. MASTER DEVICE: The end-effector twist is related to joint velocities through the geometric Jacobian Jee(q).
- III. MASTER DEVICE: The external wrench required for admittance control is estimated rather than measured with force/torque sensors.
- III. MASTER DEVICE: The desired end-effector pose follows admittance dynamics driven by the estimated external wrench and filtered feedback wrench.
- III. MASTER DEVICE: An iterative inverse-kinematics algorithm with null-space projection converts the desired end-effector pose into joint angles while managing redundancy.
D. Master Command Generation
Master pose deviations are converted into a deadbanded twist command, with control-mode gating selecting the transmitted translational and rotational components.
- D. Master Command Generation: The master twist command ξc is computed from the end-effector pose deviation relative to its neutral configuration.
- D. Master Command Generation: A linear deadband mapping generates nonzero twist only when pose deviation exceeds the specified component-wise thresholds.
- D. Master Command Generation: The mapping uses a positive diagonal gain matrix KM and a six-component deadband vector l.
- D. Master Command Generation: The twist command is selectively gated so only relevant translational and rotational components reach the slave in each control mode.
- D. Master Command Generation: The hierarchical safety filter then adjusts ξc first for interaction-consistent tracking errors and subsequently for thrust-limit compliance.
- D. Master Command Generation: The interaction and saturation filters are formulated as quadratic programs subject to control barrier function constraints and generate corresponding haptic wrenches.
A. Preliminary: Affine Mappings in Aerial Manipulator Dynamics
The aerial manipulator’s control and actuator allocation are expressed as affine mappings from desired spatial acceleration to rotor thrust magnitudes and tilt angles.
- A. Preliminary: Affine Mappings in Aerial Manipulator Dynamics: The closed-loop error dynamics are derived before formulating the interaction and saturation safety filters.
- A. Preliminary: Affine Mappings in Aerial Manipulator Dynamics: The control wrench is locally affine in the desired spatial acceleration ˙ξd.
- A. Preliminary: Affine Mappings in Aerial Manipulator Dynamics: Actuator allocation maps the control wrench into planar thrusts for each rotor using the pseudoinverse of the thrust allocation matrix.
- A. Preliminary: Affine Mappings in Aerial Manipulator Dynamics: For each rotor, thrust magnitude Ti and angle θi are represented through the allocation mapping, with B ∈R6×2nr.
- A. Preliminary: Affine Mappings in Aerial Manipulator Dynamics: The resulting thrust vector for each rotor is affine in the desired spatial acceleration ˙ξd.
- A. Preliminary: Affine Mappings in Aerial Manipulator Dynamics: The interaction filter modifies ξc into ξ1, whose translational and angular components are decision variables subject to interaction constraints.
- A. Preliminary: Affine Mappings in Aerial Manipulator Dynamics: The desired pose is obtained by integrating ξc from the current pose, defining the position and attitude tracking errors used by the filter.
WRT B1
The interaction filter models position and attitude tracking errors and enforces their admissible bounds through soft control barrier function constraints in a quadratic program.
- WRT B1: Position and attitude errors evolve according to the manipulator’s translational and angular-velocity errors.
- WRT B1: The interaction-filter decision variables v1 and Ω1 enter the formulation through the position and attitude error dynamics.
- WRT B1: Soft control barrier functions enforce admissible position-error bounds while allowing constraint relaxation through slack variables.
- WRT B1: The position-error barrier constraints are rearranged with respect to v1 to constrain the interaction-filter command.
- WRT B1: An attitude-error radius θR defines the admissible orientation deviation used by the corresponding control barrier function.
- WRT B1: The position and attitude barrier inequalities are combined to construct the interaction filter as a control-barrier-function quadratic program.
C. 2nd Stage: The Saturation Filter
The saturation filter modifies the commanded jerk to enforce motor thrust limits while staying as close as possible to the nominal motion command.
- The saturation filter prevents motor saturation by adjusting the interaction filter’s output with a jerk input that complies with motor thrust limits.
- The nominal desired acceleration is defined from the current desired twist and a time constant as a reference for the saturation filter.
- The filter computes a jerk that minimally deviates from the nominal jerk while enforcing thrust constraints.
- Thrust feasibility is enforced through a hard bound on each rotor’s thrust magnitude.
2) Filter Design:
The filter formulates thrust-limit safety as control-barrier constraints and solves a quadratic program to obtain a safe jerk correction.
- Filter Design: The thrust-limit control barrier function enforces nonnegative safety values over time for each rotor.
- Filter Design: The control-barrier condition is converted into a linear inequality in the jerk decision variable.
- Filter Design: A quadratic program computes the saturation-filter jerk subject to the resulting thrust-safety inequality.
- Filter Design: The optimal jerk correction is used to update the desired acceleration.
D. Haptic Feedback Wrench Generation
The framework converts safety-filter corrections into acceleration cues and renders them as haptic wrenches, combining interaction and saturation feedback with saturation emphasized.
- Each filter correction is expressed as an acceleration cue and rendered through the mapping G from spatial acceleration to body-frame control wrench.
- The interaction filter’s velocity-level adjustment is converted into a feedback wrench using an acceleration cue and time constant.
- The saturation filter’s jerk-level modification is likewise converted into a feedback wrench.
- The interaction and saturation feedback wrenches are linearly summed into the total feedback wrench.
- User-defined positive diagonal gains weight the two feedback channels, with K2 > K1 emphasizing saturation feedback.
V. EXPERIMENTAL RESULTS
Experiments with a hexarotor-based omnidirectional aerial manipulator compare the proposed haptic feedback against momentum-based estimation and test external displacement and manipulation tasks. The proposed method remains stable in free flight, produces directionally aligned feedback, and supports multiple tasks while avoiding feedback under bounded hovering error.
- Experimental Setup: The experimental platform uses a hexarotor-based omnidirectional tiltrotor aerial manipulator with visual feedback and a rigidly attached 0-DoF manipulator.
- Experimental Setup: The nominal position-error bound is 0.025 m per axis, while the attitude-error bound is 5° and is relaxed to 10° for the omnidirectionality test.
- Comparative Experiments: During free flight, momentum-based feedback exhibits unintended hovering drift, whereas the proposed method remains stable.
- Comparative Experiments: When the aerial manipulator is externally displaced, the master end-effector moves with the deviation and the feedback wrench aligns with its direction.
- Comparative Experiments: The proposed method generates no wrench feedback during nominal hovering when tracking error remains within the allowable bound.
- Comparative Experiments: Cart pulling, plug pulling, and push-and-slide produce feedback wrenches aligned with the expected physical-interaction directions.
- Comparative Experiments: The framework extends haptic perception to external disturbances such as wind gusts or impacts outside manipulation tasks.
C. Validation of Proposed Feedback
Experiments validate directionally coherent haptic feedback during induced deviations and diverse manipulation tasks, while the saturation filter prevents motor saturation and supports stable corrective actions.
- The proposed wrench feedback aligns with the direction of externally induced pose deviations, conveying interaction cues intuitively to the operator.
- Cart pulling, plug pulling, and push-and-slide tasks produce feedback wrenches aligned with expected physical-interaction directions and are completed successfully.
- The omnidirectional aerial manipulator hooks, lifts, and transports a box approached near-vertically between two tall obstacles.
- With the saturation filter enabled, corrective haptic forces indicate desired-position adjustments toward −x, −y, and −z.
- The saturation filter alleviates the load on the motor closest to saturation while preserving flight stability, preventing motor saturation and destabilization.
- Hardware experiments show responsive, directionally consistent interaction feedback, while the ablation confirms improved stability and informative corrective feedback from saturation filtering.