PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning to Leverage Compliance: A Policy-Admittance Learning Framework for Robotic Insertion

Chongren Wang, Minghe Li, Honghua Dai, Zhicheng Lin, Shiyang Wei, Xiaokui Yue

arXiv:2609.31439v1cs.RO

TL;DR

Policy learning and compliant control do not by themselves ensure coordination when a policy continues pushing as the controller yields. LeCo turns high-rate contact interaction under fixed admittance into policy-transition rewards; across four real connector-assembly tasks, the paper reports improved performance and reduced contact loads.

  • Problem

    Policy learning and compliant control can remain uncoordinated, with sustained loading and limited progress, while transition-level feedback must distinguish sustained opposition from brief high-force events.

    Source: §I · p. 1

  • Method

    LeCo aggregates high-rate contact records into policy-transition rewards combining an integrated conflict cost and a directional high-force tail cost under fixed admittance.

    Source: §I · p. 2

  • Results

    Across four real connector-assembly tasks, LeCo improves autonomous acquisition and reduces contact loads; ablations identify complementary roles for tail and conflict shaping.

    Source: §V · p. 10

  • Takeaways & Limitations

    The findings support using an existing compliant controller as both an execution mechanism and a source of learning supervision.

    Source: §V · p. 10

    Evaluation uses fixed fixtures and limited sets; transfer across robots and end effectors and robustness to disturbances, workpiece motion, friction, and mating-clearance changes remain unevaluated under controlled variations.

    Source: §V · p. 10

Abstract

from arXiv · show

Policy learning and compliant control offer a promising route to reliable autonomous assembly under pose errors and contact uncertainty. However, combining them does not ensure coordination: the policy may continue pushing against contact while the controller yields, producing sustained loading with limited progress. To address this problem, we propose LeCo (Leverage Compliance), a policy-admittance learning framework that guides a visual policy through execution-time interaction under fixed admittance. A multirate feedback mechanism aggregates high-rate contact-interaction records into policy-transition rewards. An integrated conflict cost then characterizes sustained policy-loading/controller-unloading opposition, while a directional high-force tail cost captures continued-loading events within a transition. Together with task completion, these costs encourage the policy to leverage compliance with less unproductive loading. We evaluate LeCo on four real connector-assembly tasks, obtaining an aggregate success rate of 94%. Across tasks, mean successful-trial resultant-force and torque peaks decrease by approximately 30% and 64% relative to the comparison baseline. Reward ablation further shows that adding conflict shaping reduces median successful-trial contact-conditioned conflict density by approximately 53%. These results support learning to leverage fixed compliance by turning multirate policy-admittance interaction into complementary reward signals for effective, lower-load insertion.

I. INTRODUCTION

Reliable assembly must balance task progress with compliant contact under pose errors and contact uncertainty. LeCo uses multirate interaction feedback and complementary costs to guide a visual policy under fixed admittance.

A. Learning Assembly Policies

  • Reliable assembly requires advancing a part toward its seat while correcting misalignment under pose errors and contact uncertainty.Excessive resistance prevents progress, whereas insufficient engagement prevents completion.Source: §I · p. 1

I. INTRODUCTION

  • Task completion alone cannot distinguish necessary brief correction from repeated command opposition, and high-force events can occur during either continued loading or withdrawal.Final observations may miss earlier opposition, while interval averages can dilute brief loading events and maxima omit duration.Source: §I · pp. 1–2
  • LeCo combines task completion with interaction costs that target sustained policy-loading/admittance-unloading opposition and brief high-force loading.High-rate contact records are aggregated over transition-associated intervals to provide policy-rate feedback.Source: §I · p. 2

A. Learning Assembly Policies

  • Assembly-policy research includes geometric modeling, simulation transfer, force-conditioned behavior, demonstrations, and real-robot policy learning with human corrections.LeCo addresses the contact objective within policy learning by asking whether a successful reference cooperates with the compliant response encountered during execution.Source: A. Learning Assembly Policies · p. 2

B. Learning Compliant Contact Behavior

  • Compliant-manipulation methods differ in what they learn; LeCo keeps admittance parameters fixed and shapes the policy reference using interaction with the compliant response.Prior approaches learn mechanical response, contact commands or behaviors, or combine residual learning with compliant control.Source: B. Learning Compliant Contact Behavior · p. 2

C. Reference Adaptation and Multirate Feedback

LeCo treats policy motion and fixed-admittance response as a shared closed-loop learning problem. It uses fast-loop interaction records to shape transition-level rewards while retaining separate task decisions and rapid contact response.

C. Reference Adaptation and Multirate Feedback

  • LeCo uses fast-loop records as reward supervision: integration represents sustained opposition, while directional maximization retains short loading events.Unlike hard-constraint approaches, it shapes a task objective toward lower interaction costs rather than imposing hard force constraints.Source: C. Reference Adaptation and Multirate… · p. 2

III. PROBLEM FORMULATION AND METHODS

  • The policy selects actions from observations while admittance adjusts execution using wrench feedback; high-rate records within each transition become conflict and tail costs.The policy–admittance interaction is formulated as a partially observable Markov decision process under fixed-admittance closed-loop dynamics.Source: §III · p. 2, A. Problem Formulation and System… · p. 2, Figure 2 · p. 3

A. Problem Formulation and System Overview

  • Observations combine multiview RGB images with robot state, and actions specify scaled Cartesian motion increments across translation and rotation.The robot state includes reset-relative TCP pose and velocity, force, and torque; the objective combines task completion with reduced sustained conflict and transient high-force loading.Source: A. Problem Formulation and System… · p. 3
  • A common transition interval aligns policy and contact-control updates, with configured periods of 0.1 s and 0.01 s, respectively.An uninterrupted synchronous interval contains ten admittance updates, and state evolution and both interaction costs use the same interval.Source: A. Problem Formulation and System… · p. 3

B. Multirate Policy–Admittance Execution

LeCo computes interaction costs from policy and admittance commands together with wrench measurements. Its implementation accounts for reference mapping, wrench coordinates, and variable-rate recording windows.

B. Multirate Policy–Admittance Execution

  • The policy reference is mapped from displacement to twist before workspace and velocity limiting, and workspace-clipped targets are converted back using the configured policy period.The conversion uses Tπ = 0.1 s rather than measured interval duration.Source: B. Multirate Policy–Admittance Execution · p. 3
  • The admittance residual uses fixed diagonal virtual-inertia, damping, and axis-selection matrices with an axis-wise deadband; its bounded state carries across samples and window boundaries.The residual state is zeroed only at episode reset, while lateral and rotational commands combine policy and admittance components.Source: B. Multirate Policy–Admittance Execution · pp. 3–4
  • Hardware recording windows use timestamped controller-buffer snapshots rather than enforcing exactly ten samples or exact alignment with one action.After an action block of at most ten inner ticks, the policy reference is set to zero while admittance continues; these rewards approximate synchronous transition-level costs.Source: B. Multirate Policy–Admittance Execution · p. 4
  • Interaction power uses the wrench and twists expressed consistently at a common TCP reference point, with the insertion-end wrench obtained by shifting torque from the TCP.Positive loading power is defined using the paper's loading sign convention.Source: B. Multirate Policy–Admittance Execution · p. 4

C. Multirate Analytic Cooperation Reward

The conflict cost measures sustained command-level opposition between policy loading and admittance unloading, while temporal integration preserves its duration. Its force projection uses the fixed robot-base xy plane, and the integral is not interpreted as mechanical dissipation or wasted work.

1) Sustained policy–admittance conflict:

  • Fig. 3(a) illustrates that integrating shared command power preserves conflict duration across a policy window.This aggregation complements the maximum-based tail term, which retains brief peaks attenuated by averaging.Source: Figure 3 · p. 4
  • The conflict measure attributes opposition only to shared policy-loading and admittance-unloading power; withdrawal, orthogonal correction, or an inactive component contributes none.The translational projection is fixed to the robot-base xy plane, while the rotational cost uses all three torque channels.Source: 1) Sustained policy–admittance conflict · p. 4
  • Integrating filtered conflict power over the policy interval yields an energy-unit quantity intended to represent persistent interaction, not mechanical dissipation or wasted work.Filtering is used to emphasize persistence rather than measurement spikes.Source: 1) Sustained policy–admittance conflict · p. 4

2) Directional high-force tail cost:

The directional tail cost complements the conflict integral by retaining the strongest gated continued-loading event within a policy interval. It is zero during policy unloading or motion orthogonal to the force.

2) Directional high-force tail cost:

  • Raw lateral force sets a gate that is zero below 2 N, changes smoothly between 2 N and 3 N, and equals one above 3 N; the tail term uses positive raw policy-loading power under high lateral force.Source: 2) Directional high-force tail cost · p. 4
  • Policy unloading or motion orthogonal to the force makes the tail term zero even when the force gate is open.The sample maximum is bounded by qmax = 10, giving a per-reward tail-penalty magnitude bound of 1.0 at ηF = 0.1.Source: 2) Directional high-force tail cost · pp. 4–5
  • The tail term takes the maximum gated policy-loading sample over the interval, preserving a brief high-force event that the conflict integral may underrepresent.The gate uses raw lateral-force magnitude, which retains fast contact transients.Source: 2) Directional high-force tail cost · pp. 4–5

D. Controller-Aware Learning

LeCo stores task reward and interaction costs at policy-transition rate while admittance remains continuous across intervals. The learner uses online experience together with demonstration and intervention transitions.

D. Controller-Aware Learning

  • At each policy transition, the stored reward combines classifier-based task completion, a time penalty, the conflict cost, and the directional tail cost.Success terminates an episode; otherwise, all four tasks use a 150-policy-step timeout.Source: D. Controller-Aware Learning · p. 5
  • The interaction costs summarize the same interval, but the resulting reward characterizes joint closed-loop evolution rather than the isolated causal effect of the current action.Earlier contact and controller memory remain part of the evolving state.Source: D. Controller-Aware Learning · p. 5
  • The learner stores executed actions in online replay, adds corrected transitions to demonstration/intervention replay, and samples mixed batches for policy and critic updates.Online experience captures current-policy contact behavior, while demonstrations and corrections provide examples of completion and contact adjustment.Source: D. Controller-Aware Learning · p. 5

IV. EXPERIMENTS

The experiments assess LeCo on four real connector-assembly tasks and compare it with learning baselines under shared evaluation conditions. They measure contact load, autonomous training success, and execution efficiency.

IV. EXPERIMENTS

  • The experiments ask whether LeCo reduces contact loads while improving success and efficiency, whether interaction rewards reduce sustained opposition, and how each cost affects acquisition and contact quality.The evaluation includes four real connector-assembly tasks.Source: §IV · p. 5

1) Tasks and platform:

  • The four tasks span aviation-connector insertion, Anderson-head insertion, Anderson-side interlocking, and ATX 20+4-pin housing assembly, with differing alignment and contact demands.The task set tests the balance between necessary advancement and unproductive loading.Source: 1) Tasks and platform · pp. 5–6, Figure 4 · p. 6

2) Baselines and metrics:

  • LeCo uses a fixed six-axis admittance controller at 10/100-Hz policy/admittance rates, while HIL-SERL and ConRFT use no additional admittance layer.The comparison shares the low-level controller, demonstrations, visual layouts, action spaces, reset seeds, and operator.Source: 1) Tasks and platform · p. 6, 2) Baselines and metrics · p. 6
  • Evaluation measures lateral-force peaks, autonomous success over the last ten training episodes, intervention steps, time to a success level, and successful evaluation length.Training statistics use the last ten complete episodes, with an expanding prefix before ten are available.Source: 2) Baselines and metrics · p. 6

B. Comparison With Learning Baselines

LeCo reduced training contact loads and reached autonomous learning milestones sooner, then maintained task success with lower contact peaks after training, though it did not require the fewest steps on every task.

1) Training contact loads and learning behavior:

  • 2.43 N versus 9.12 N for ConRFT and 8.54 N for HIL-SERL was LeCo's final displayed ten-episode median lateral-force peak on Anderson-side interlocking, approximately 73% below ConRFT.The displayed training statistics include assisted and failed episodes.Source: 1) Training contact loads and learning… · p. 6, Figure 5 · p. 7
  • 23.7 min versus ConRFT's 46.0 min was LeCo's time to first reach 80% autonomous success on aviation insertion, approximately 48% less online time.LeCo also reached the threshold first on Anderson-head and ATX; only LeCo reached it on Anderson-side within the displayed window.Source: 1) Training contact loads and learning… · p. 6
  • Lower training loads did not come from abandoning contact or relying on more human takeover; LeCo acquired autonomous insertion earlier and needed fewer corrections late in training.The combined force-peak, intervention-ratio, and autonomous-success results indicate joint improvement in contact quality and learning efficiency.Source: 1) Training contact loads and learning… · p. 6, Figure 5 · p. 7

2) Post-training autonomous performance:

  • LeCo combines higher success, fewer steps, and lower contact peaks on aviation and ATX, while lowering force and torque peaks on Anderson tasks despite requiring more steps there.Equal success on Anderson-head does not imply equal contact quality; LeCo has lower lateral-force and torque peaks.Source: Table II · p. 7, 2) Post-training autonomous performance · pp. 7–8

C. Ablation

The MAF36A ablation separates task-only, conflict-shaped, tail-shaped, and combined rewards under fixed admittance. LeCo performs best on the combined completion, efficiency, and accumulated-interaction criteria.

1) Reward variants and evaluation metrics:

  • The MAF36A 2 × 2 ablation compares task-only A0, conflict-shaped A1, tail-only A2, and LeCo with both costs while holding admittance and other learning conditions fixed.All variants share the time penalty, observations, action space, demonstrations, and fixed admittance controller.Source: 1) Reward variants and evaluation… · p. 8
  • Mechanical metrics include every attempt because success-conditioned statistics can hide long-contact failures, while all-attempt peak statistics can reward policies that barely touch the task.The conflict measure captures accumulated opposition between policy and admittance commands; conflict per contact time and rotational command cancellation also assess coordination.Source: 1) Reward variants and evaluation… · p. 8

2) The combined reward gives the best overall performance:

  • LeCo achieved 19/20 deterministic successes, a 33.0-step median among successful trials, and the lowest median and P90 lateral-force exposure, torque exposure, and accumulated conflict across all attempts.These outcomes give LeCo the best combined completion, efficiency, and contact quality among the four reward variants.Source: 2) The combined reward gives the best… · p. 8

3) Complementary roles of tail and conflict shaping:

Across four real connector-assembly tasks, LeCo combines tail and conflict shaping to support autonomous acquisition while reducing accumulated interaction loads. The ablations show complementary roles for the two costs, with benefits in successful-trial opposition and complete-attempt exposure.

3) Complementary roles of tail and conflict shaping:

  • Tail-only A2 reaches an 80% autonomous-success training window at approximately 29 min versus 46 min for A0, and improves frozen-policy success from 15/20 to 18/20.A2 also reduces failures from five to two and P90 lateral exposure from 41.90 to 8.80 N s.Source: 3) Complementary roles of tail and… · pp. 8–10
  • Compared with A2, LeCo reduces successful-trial median contact-conditioned conflict density from 1.29 to 0.60 s−1 and contact duration from 3.91 to 2.26 s.The reduction is mainly rotational conflict under reference-energy scaling; lateral-conflict profiles largely overlap.Source: Figure 8 · p. 9, 3) Complementary roles of tail and… · p. 10

4) Accumulated interaction across complete attempts:

  • Across complete attempts, LeCo reduces median lateral-force exposure from 6.17 to 3.64 N s, torque exposure from 1.38 to 0.81 N m s, and accumulated conflict from 5.03 to 1.38.All three P90 values also decrease, although LeCo is not best on every instantaneous peak metric.Source: Table IV · p. 9, 4) Accumulated interaction across… · p. 10

3) Complementary roles of tail and conflict shaping:

  • Adding the tail term to A1 moves the first 80% training window from approximately 43 to 24 min, raises frozen-policy success from 12/20 to 19/20, and reduces median successful length from 88.5 to 33.0 steps.This comparison adds the tail term while conflict shaping is present.Source: 3) Complementary roles of tail and… · p. 9

V. CONCLUSION

  • LeCo keeps controller parameters, policy interface, and deployment structure unchanged; its conflict integral captures sustained opposition, while its directional tail cost retains brief high-force loading events.The conflict term concerns policy loading against admittance unloading; the tail term preserves continued-loading events.Source: §V · p. 10
  • Across four real connector tasks, LeCo improves autonomous acquisition and reduces contact loads; combined shaping shortens successful trajectories and lowers accumulated force, torque exposure, and command conflict.The conclusion does not claim lower instantaneous loads at every moment.Source: §V · p. 10

APPENDIX A PHYSICAL INTERPRETATION OF INTERACTION COSTS AND

The appendix interprets the interaction costs as duration-sensitive conflict and worst continued-loading events, then states conditions for bounded rewards and convergence of fixed-policy evaluation.

POLICY EVALUATION

  • As sampling becomes finer, the interval sum approaches the conflict-power integral, while the maximum sampled gated, capped tail term approaches its interval supremum.These reductions preserve conflict duration and the strongest directional high-force event, respectively.Source: POLICY EVALUATION · p. 10
  • The resulting bounds describe expected conflict exposure and continued-loading high-force events, while the directional tail term leaves withdrawal and force-orthogonal corrections unpenalized.The ideal weighted-cost bound does not bound each cost component separately.Source: POLICY EVALUATION · p. 10
  • With bounded reachable wrench, commanded twist, and policy-window duration, the conflict term and total reward are bounded; the fixed-policy Bellman operator remains a γ-contraction for bounded rewards.At γ = 0.97, exact fixed-policy Bellman iteration converges to a unique policy value function.Source: POLICY EVALUATION · p. 11

Related papers

Loading 2609.31439v1…