PaperlayerView exact revision on arXiv

Source-linked AI summary

Experience-Based Feasibility-Aware Generative Adversarial Imitation from Observation under Embodiment Mismatch

Yoshiki Takebayashi, Giovanni Perantoni, Hikaru Sasaki, Matteo Saveriano, Takamitsu Matsubara

arXiv:2610.01171v1cs.RO

TL;DR

Embodiment mismatch can make state-only human demonstrations infeasible for robots, limiting their direct use for imitation. EF-GAIfO estimates feasibility from robot experience and weights demonstrations accordingly; evaluations report more discriminative feasibility estimates, improved task results, and successful real-robot execution.

  • Problem

    Human demonstrations may contain transitions that a robot cannot reproduce because of embodiment mismatch, potentially encouraging infeasible motions and degrading task performance.

    Source: §I · p. 1

  • Method

    EF-GAIfO estimates demonstration feasibility from robot-experienced state transitions using reconstruction error, then incorporates feasibility weights into discriminator training.

    Source: §I · pp. 1–2

  • Results

    EF-GAIfO’s feasibility estimates became more discriminative with accumulated experience, and it achieved higher success rates and smaller Hausdorff distances than both baselines in multi-speed and multi-path settings.

    Source: 2) Results · p. 5

  • Takeaways & Limitations

    With human-collected demonstrations, EF-GAIfO achieved high simulation success rates and successful execution on a real quadruped robot.

    Source: §VI · p. 7

Abstract

from arXiv · show

With the increasing use of robot-free demonstration interfaces that provide state trajectories without action labels, imitation from observation has become a promising approach for learning robot behaviors from human demonstrations. However, due to differences in embodiment and dynamics between humans and robots, demonstrated human motions may not be feasible for the robot, potentially degrading policy performance. In this study, we propose Experience-Based Feasibility-Aware Generative Adversarial Imitation from Observation (EF-GAIfO), which estimates the feasibility of state-only demonstrations from the robot's own experience rather than relying on explicit dynamics models or large prior exploration datasets. A key feature of EF-GAIfO is that the notion of feasibility evolves with policy learning: as the policy improves and the robot experiences a broader range of state transitions, the feasible region is progressively expanded, allowing additional demonstrations to be incorporated into learning. This enables feasibility-aware imitation that adapts to the current stage of policy learning, rather than relying on a pre-designed feasibility criterion. We validate the effectiveness of EF-GAIfO on a locomotion task in simulation and on a real quadruped robot performing a object-reaching-and-grasping task.

I. INTRODUCTION

Robot-free interfaces provide state-only human demonstrations for imitation from observation, but embodiment mismatch can make some demonstrated transitions infeasible for a robot. EF-GAIfO estimates feasibility from the robot’s own experience and uses those estimates to reduce the influence of infeasible demonstrations.

I. INTRODUCTION

  • Robot-free interfaces let users demonstrate with hand movements, typically yielding state trajectories without robot action labels, for which imitation from observation is suitable.The cited approaches include video-based, infrastructure-based, and onboard-tracking interfaces, [2],,, [5].Source: §I · p. 1
  • Embodiment mismatch can make demonstrated transitions infeasible for the robot, potentially encouraging unreproducible motions and causing falls or degraded task performance.The concern arises when such demonstrations are directly used for imitation from observation.Source: §I · p. 1
  • Model-based feasibility methods predict executability using known or learned robot dynamics, but coupled whole-body dynamics, contact, and balance constraints make accurate modeling difficult.Source: §I · p. 1
  • Robot experience provides an embodiment-specific reference for feasibility without an explicit dynamics model, and the covered region expands as experience accumulates.This allows more demonstration transitions to receive high feasibility estimates over policy learning.Source: §I · p. 1, Figure 1 · p. 1
  • EF-GAIfO estimates demonstration feasibility from robot experience and reduces the influence of transitions with low estimated feasibility.It uses state-transition reconstruction error and incorporates the resulting weights into discriminator training.Source: §I · pp. 1–2

A. Imitation Learning from Observation

  • Prior imitation-from-observation work addresses observation-only or partial demonstrations but does not fully address feasibility problems from demonstrator–robot physical differences.EF-GAIfO introduces experience-based feasibility estimation to reduce the influence of robot-infeasible transitions.Source: A. Imitation Learning from Observation · p. 2

B. Imitation Learning from Imperfect and Infeasible Demonstrations

Prior work addresses imperfect demonstrations through quality estimation and feasibility-aware methods, but approaches may require labels, dynamics models, or prior robot data. GAIfO matches state-transition distributions with a discriminator and optimizes the policy using discriminator-derived rewards.

B. Imitation Learning from Imperfect and Infeasible Demonstrations

  • Imperfect demonstrations can contain suboptimal actions, mistakes, noise, or robot-unexecutable motions; prior quality methods use confidence labels or negative-risk terms, [17].Source: B. Imitation Learning from Imperfect… · p. 2
  • Other quality approaches distinguish expert from non-expert data,, rank demonstrations to extrapolate rewards, extend ranking with generator experience, or learn adversarial per-demonstration weights.Source: B. Imitation Learning from Imperfect… · p. 2
  • Prior feasibility-aware methods address dynamics mismatch, potentially non-reproducible behavior, or hand-interface demonstrations using dynamics models or multimodal feedback.Examples include approaches using robot dynamics and visual and haptic feedback [25].Source: B. Imitation Learning from Imperfect… · p. 2
  • The proposed method estimates feasibility from state transitions collected online during policy learning, without confidence labels, an explicit dynamics model, or a separate prior exploration dataset.Source: B. Imitation Learning from Imperfect… · p. 2

III. GENERATIVE ADVERSARIAL IMITATION LEARNING FROM OBSERVATION (GAIFO)

  • GAIfO learns a policy to reproduce state-only demonstrations through reinforcement learning, using a discriminator to distinguish policy-generated from expert state transitions.Policy optimization can use rewards derived from the discriminator output, such as with PPO [26].Source: §III · p. 2
  • EF-GAIfO uses robot experience to estimate feasibility of state-only demonstration transitions and incorporates feasibility weights into discriminator training.Source: Figure 2 · p. 3

IV. EXPERIENCE-BASED FEASIBILITY-AWARE GAIFO

EF-GAIfO estimates demonstration feasibility by comparing state transitions with those reconstructed from the robot’s policy-learning experience. It uses reconstruction error to weight demonstrations during discriminator training while jointly updating the policy, estimator, and discriminator.

IV. EXPERIENCE-BASED FEASIBILITY-AWARE GAIFO

  • EF-GAIfO jointly updates the policy, feasibility estimator, and discriminator, using estimated feasibility to reduce infeasible demonstrations’ influence during discriminator training.The framework first estimates transition feasibility from similarity to robot-experienced transitions.Source: §IV · p. 3

A. Feasibility Estimation

  • The estimator assumes feasible demonstrations resemble robot-experienced transitions and trains a reconstruction model on state transitions collected during policy learning.For transition z_t = (s_t, s_{t+1}), the model R produces a reconstructed transition.Source: A. Feasibility Estimation · p. 3
  • The reconstruction model is trained by minimizing reconstruction loss over a state sequence, whose length is T.Source: A. Feasibility Estimation · p. 3
  • The learning procedure initializes the policy, value function, discriminator, reconstruction model, and experience buffer, then collects robot transitions and updates the policy with PPO.Source: A. Feasibility Estimation · p. 3
  • Transition reconstruction error uses the worse reconstruction error across its two consecutive states, so either poorly reconstructed state raises the transition error.The error is calculated across state dimensions using original and reconstructed states.Source: A. Feasibility Estimation · p. 3
  • The feasibility weight maps reconstruction error to a value from 0 to 1 using lower and upper thresholds and clipping.The algorithm computes reconstruction error and feasibility weights from experience batches, then updates the discriminator with those weights.Source: A. Feasibility Estimation · p. 3

B. Feasibility-weighted Objective

EF-GAIfO weights demonstration data in the discriminator objective according to estimated feasibility. This directs policy learning toward demonstrations supported by the robot’s experience.

B. Feasibility-weighted Objective

C. Learning Process

EF-GAIfO iteratively uses robot experience to update the policy and reconstruction model, then weights demonstrations for discriminator training.

C. Learning Process

  • Each iteration collects transitions from the current policy and updates the policy using rewards.Source: C. Learning Process · p. 3
  • The collected experience also updates the reconstruction model, which evaluates demonstration samples and computes their feasibility weights.Source: C. Learning Process · p. 3
  • The discriminator is updated using feasibility-weighted demonstrations and robot experience.Source: C. Learning Process · p. 3
  • The feasibility-weight thresholds use demonstration reconstruction errors, with percentile parameter q gradually increased during training.Source: C. Learning Process · p. 4

V. EXPERIMENTS

The experiments address whether experience-based estimates distinguish feasible transitions as experience accumulates, improve imitation across feasibility mismatches, and support learning from human demonstrations on a robot task.

V. EXPERIMENTS

  • RQ1 asks whether experience-based feasibility estimation distinguishes feasible from infeasible demonstration transitions as robot experience accumulates.Source: §V · p. 4
  • RQ2 asks whether feasibility weighting improves imitation performance across different forms of feasibility mismatch.Source: §V · p. 4
  • RQ3 asks whether EF-GAIfO can learn from human-collected state-only demonstrations and enable successful execution on a real robot.Source: §V · p. 4

1) Settings:

The point-mass experiments test feasibility discrimination and imitation under speed and path mismatches. EF-GAIfO outperforms both baselines on task success and trajectory distance, while its feasibility estimates become more discriminative with accumulated experience.

1) Settings:

  • The multi-speed setting tests selection of feasible speeds along a fixed path, whereas the multi-path setting tests selection among mixed trajectory modes.Source: 1) Settings · p. 4
  • In the multi-speed setting, 20 nearly same-path demonstrations comprise 10 slower trajectories within velocity limits and 10 faster trajectories exceeding them.Figure 3 marks feasible and infeasible trajectories and uses color intensity to indicate execution speed.Source: 1) Settings · p. 4, Figure 3 · p. 4
  • The multi-path dataset contains 50 demonstrations across five paths, of which two are configured to be feasible under the agent’s speed limits.Source: 1) Settings · p. 4
  • Each method was trained for 3000 iterations across 20 random seeds and evaluated over 10 episodes per seed in each setting.Source: 1) Settings · p. 4

2) Results:

  • EF-GAIfO achieved the highest task success rates and smallest Hausdorff distances versus WGAIfO and GAIfO in both point-mass settings.Source: 2) Results · p. 4, Table I · p. 5
  • As training progresses, EF-GAIfO assigns higher evaluation values to feasible demonstrations while infeasible demonstrations remain low; WGAIfO lacks the same progressive separation.The results are shown for both multi-speed and multi-path settings; Figure 4 compares estimates at 100 and 3000 iterations.Source: Table II · p. 5, 2) Results · p. 5, Figure 4 · p. 5
  • In the multi-path setting, EF-GAIfO’s results suggest it can selectively imitate feasible trajectory modes.Source: 2) Results · p. 5

1) Settings:

The quadruped task evaluates learning from human-collected demonstrations for reaching and picking up objects. The setup includes challenging motion variations and a hierarchical policy for locomotion and task commands.

1) Settings:

  • The Go2 starts at (0, 0) m and must reach and lift objects at three fixed target positions; success requires grasping and lifting before termination.An episode also terminates when the maximum episode length is reached or the robot base contacts the ground.Source: 1) Settings · p. 5
  • Simulation success rates are averaged over five policies trained with different random seeds, each evaluated in 100 trials.The table covers three fixed target positions and a random target setting.Source: Table III · p. 6
  • Demonstrations include fast reaching and large pitch motions, with 30 trajectories recorded at 100 Hz for approximately 10 s each.Ten trajectories were collected for each of the three target positions; the visualized demonstrations vary in planar position and pitch motion.Source: 1) Settings · pp. 5–6, Figure 7 · p. 6
  • A pretrained low-level policy controls locomotion, while a learned high-level policy commands navigation, body pitch, and gripper motion.The high-level commands include forward and lateral velocity, yaw rate, pitch, and gripper open/close.Source: 1) Settings · p. 6

2) Results:

EF-GAIfO outperformed GAIfO and WGAIfO in simulated and real-world object-reaching-and-picking evaluations. Its feasibility estimates increasingly downweighted large-pitch transitions as training progressed.

2) Results:

  • EF-GAIfO achieved higher success rates than GAIfO and WGAIfO across all simulated fixed and random target settings, generalizing to nearby target positions.Representative executions show EF-GAIfO completing pickup, while WGAIfO’s excessive pitch causes ground contact and GAIfO exhibits gripper–object misalignment.Source: 2) Results · p. 6, Figure 8 · p. 6
  • As training progressed, EF-GAIfO assigned lower feasibility weights to large-pitch demonstration transitions, suppressing motions difficult for the robot to reproduce.The weights are shown at 100 and 3000 training iterations.Source: 2) Results · p. 6, Figure 9 · p. 6

3) Real-World Experiment:

  • 70–80% success rates across fixed and random target settings were achieved by EF-GAIfO on the real Go2, with stable locomotion and successful object pickup.The evaluation used ten trials per setting; the reported examples show EF-GAIfO succeeding while WGAIfO and GAIfO fail during grasping.Source: Table IV · p. 7, 3) Real-World Experiment · p. 7, Figure 10 · p. 7

VI. CONCLUSIONS AND FUTURE WORK

The conclusion reports that EF-GAIfO improves feasibility discrimination as robot experience accumulates and performs across different forms of feasibility mismatch. Its current estimator mainly captures embodiment-related motion feasibility.

VI. CONCLUSIONS AND FUTURE WORK

  • EF-GAIfO achieved higher task success rates and closer trajectory matching across different forms of feasibility mismatch, including with human demonstrations and real quadruped execution.The paper presents these results as evidence of practical applicability for demonstrations with unknown feasibility.Source: §VI · p. 7
  • Feasibility estimates became more discriminative as robot experience accumulated.Source: §VI · p. 7
  • The estimator mainly captures embodiment-related motion feasibility; accounting for terrain, obstacles, object placement, and contact requires environmental context.Future work proposes conditioning feasibility estimation on robot transitions and environmental observations, including visual and depth information.Source: §VI · p. 7

Related papers

Loading 2610.01171v1…