PaperlayerView exact revision on arXiv

Source-linked AI summary

Rapid On-Robot Learning for Dynamic Manipulation Skills: Robot Juggling

Taeyoon Lee, Chunpeng Wang, Christopher G. Atkeson, Alfred A. Rizzi, Nicolas Rojas

arXiv:2608.26800v1cs.RO

TL;DR

Dynamic manipulation can remain difficult when simulation, analytical models, or demonstrations do not capture physical deployment. This paper combines regularized memory-based learning with a safety constraint so a bimanual robot can refine throws from experience, learning five three-ball juggling patterns in under five minutes despite an inaccurate prior model.

  • Problem

    Fast, intermittent-contact manipulation can be highly sensitive to sim2real discrepancies, while safe continual learning must avoid inefficient or unsafe exploration.

  • Method

    Regularized memory-based learning builds a local model from physical experience while retaining the prior for sparse regions, and a mutually reachable set constrains safe successive throws and catches.

  • Results

    The bimanual robot learned and composed five distinct three-ball juggling patterns—cascade, tennis, half-shower, shower, and box—with less than five minutes of on-robot learning.

  • Takeaways & Limitations

    Robots can continually improve high-performance behaviors from limited physical experience while building upon imperfect prior knowledge and maintaining safe transitions.

  • Takeaways & Limitations

    The remaining shower and box failures are limited by feedback latency because short side-throw flight times prevent visual feedback from correcting catches in time.

Abstract

from arXiv · show

We present an online learning framework that enables a bimanual robot to acquire diverse juggling patterns directly on physical hardware within minutes, even with a significant sim2real gap. One of the most important lessons from this work is that a model, even when far from reality, can be extremely useful for learning. This motivates a central philosophy of our approach: learning should build upon the robot's current knowledge rather than replace it. Our regularized memory-based learning puts this principle into practice by learning a local model from accumulated experience while retaining the global prior model to extrapolate where experience is sparse. This enables efficient and stable online learning from each new experience without resorting to uninformed exploration over a vast space of possible behaviors. Equally important to continual on-robot learning is safety, allowing the robot to repeatedly practice and improve in the real world. We construct a mutually reachable set that allows safe transitions between successive throws and catches, without driving either arm into a state from which its next action would require violating the robot's joint or actuator limits. Together, these ideas enable a bimanual robot with multi-fingered hands and onboard vision to safely learn and compose five canonical three-ball juggling patterns, including cascade, tennis, half-shower, shower, and box, within less than 5 minutes of real-world interaction. More broadly, this work points toward robots that build upon imperfect prior knowledge and continually refine their behavior through their own real-world experience.

1. INTRODUCTION

The paper addresses sim2real challenges in dynamic manipulation by combining prior knowledge with physical experience while enforcing safety during continual learning. This enables rapid learning of diverse juggling behaviors despite an inaccurate prior model.

  • Motivation: Small discrepancies in fast, intermittent-contact manipulation can significantly affect task performance, motivating learning directly from physical experience.Multi-fingered hands add variability through grasping, friction, compliance, impact, release timing, and actuation differences.
  • Contribution: The robot learns five distinct three-ball patterns—cascade, tennis, half-shower, shower, and box—with less than five minutes of on-robot learning.The prior model could not complete a single cycle of any pattern, yet the system improved from physical experience while retaining its prior knowledge.
  • Continual Learning: Physical experience helps the robot respond to changes in the robot, objects, and their interactions that can make previously robust behavior fail.The experiments observed day-to-day performance changes without intentional modifications to the system or balls.
  • Method: Regularized memory-based learning builds a local model from accumulated experience while retaining the prior model to guide behavior where experience is sparse.This corrects the prior where necessary without discarding useful knowledge elsewhere or exploring the full behavior space uninformed.
  • Safety: A mutually reachable set constrains successive throws and catches so the robot can practice dynamic behaviors without violating joint or actuator limits.The constraint avoids states in which a subsequent action would require exceeding physical limits.
  • Conclusion: The framework supports continual improvement from limited physical experience while safely learning high-performance behaviors.The approach combines efficient learning with safety rather than relying on conservative restrictions that exclude dynamic motions.

A. Related Work

Prior juggling systems achieved dynamic behavior through specialized mechanics, feedback, or physical learning, while this work combines multi-fingered hands with closed-loop, regularized memory-based learning. Its skill-conditioned formulation supports reuse across conditions and patterns.

  • Prior Systems: Specialized paddles and cups improve robustness by making ball contact predictable and reducing sensitivity to impact uncertainty.These mechanical designs can provide robustness without physical learning.
  • Prior Systems: Multi-fingered hands offer a more general interaction interface but expose behavior to difficult-to-model friction, compliance, impact, and release effects.These effects contribute to uncertainty in ball contact and release.
  • On-Robot Learning: Earlier work used iterative task-level learning, learned parametric models, and memory-based nonparametric methods to improve robotic juggling.Memory-based approaches retain and reuse physical experience for closed-loop behavior.
  • Related Learning: Ploeger and Peters regularize a local physical-experience model with first-order information from an analytical prior, closely paralleling this paper’s learning principle.Their method associates corrections with individual throws in an open-loop sequence, whereas this work uses reusable skills conditioned on observed state.
  • Related Learning: Skill-conditioned commands can be reused under different incoming conditions and across juggling patterns, while learning continues online during juggling.This contrasts with episodic updates to an open-loop sequence between trials.

2. RESULTS

The robot autonomously juggles five three-ball patterns and learns them within minutes of physical interaction, including rapid acquisition of cascade, tennis–half-shower–cascade, shower, and box behaviors.

  • Performance: The bimanual robot autonomously juggles cascade, half-shower, tennis, shower, and box using two multi-fingered hands and headmounted vision.The system also performs transitions between patterns.
  • Learning Speed: 53s of physical interaction is sufficient on average to learn cascade juggling, excluding manual ball-reset time.Manual resets involve a human retrieving dropped balls and returning them to the robot.
  • Learning Speed: 75s of physical interaction is sufficient on average to learn the continuous tennis–half-shower–cascade sequence, excluding manual ball-reset time.This result measures a continuous sequence rather than a single pattern.
  • Learning Speed: Approximately 30s and 60s of physical interaction bring shower and box performance, respectively, to their observed plateaus.These are the more challenging patterns in the reported learning process.

A. System Overview

The system schedules reusable juggling skills asynchronously, learns throw commands from physical experience, plans motions under a mutually reachable constraint, and continuously replans catches from visual feedback.

  • System Overview: An orchestrator asynchronously initiates predefined skill sequences according to a global wall-clock schedule that preserves the pattern’s temporal phase.Late skills do not delay later scheduled skills; subsequent motions compensate for accumulated timing error.
  • System Overview: The system terminates an attempt when the next ball cannot be detected or no feasible command solution exists.Both cases often originate from an inaccurate preceding throw, such as one placing the ball outside the reachable workspace.
  • Safety: The control system strictly enforces a precomputed Mutually Reachable Set to prevent motions that could reach joint limits before safely decelerating.This permits dynamic motions close to physical limits without overly conservative speed restrictions.
  • Throw Execution: For throws, the learner maps the current ball state, desired outcome, and experience memory to a corrected landing-position command.The planner converts this command into joint-space targets while enforcing the mutually reachable constraint.
  • Catch Execution: For catches, the learning module is bypassed while the system continuously replans the target trajectory from the latest ball-state estimate until 0.1s before catch time.The cutoff balances improved estimation from additional observations against the remaining time for arm motion.
  • Perception: Asynchronous perception tracks individual balls during overlapping flight phases and streams state information for closed-loop execution.Tracking begins after a throw-completion callback identifies the ball associated with that throw.

B. On-Robot Learning of Juggling Skills

The robot rapidly learns juggling from physical experience despite an inaccurate prior, using retained experience and prior regularization to improve performance across runs.

  • By the 8th attempt, the robot consistently learned the cascade and repeated it three consecutive times in approximately 5 minutes.The five independent trials started from scratch and included manual resets.
  • Landing accuracy improves as each observed throw is immediately added to memory, reducing resets and accelerating subsequent learning.This creates a positive feedback loop during ongoing juggling rather than only between attempts.
  • The prior could not sustain a single juggling cycle, yet it enabled learning to begin without dedicated initialization or random exploration.The prior still provided information about how task-level command changes affect outcomes.
  • Regularization balances local physical experience against the prior, with larger γ producing more stable but slower adaptation and smaller γ increasing sensitivity to sparse or noisy observations.The experiments used γ = 0.001 as a balance between these effects.

C. Reusability of Skills across Diverse Juggling Patterns

The robot transfers reusable juggling skills across patterns, rapidly adapting to new state distributions while learning remains stateful and closed-loop.

  • Skill reuse: Throwing skills shared across tennis, cascade, and half-shower allow experience from tennis to facilitate learning the other patterns.The robot executed Tennis → Half-Shower → Cascade continuously, sustaining each pattern for five cycles.
  • Skill reuse: After mastering tennis in approximately 7–8 resets, the robot achieved proficiency in half-shower and cascade within 2–3 additional runs.The subsequent patterns required additional experience because catching and landing-state distributions differed.
  • Trajectory adaptation: Figure 3 tracks ballistic flight arcs and landing locations across resets, showing tighter clustering and reduced trajectory variance over learning.Rows represent cascade, the transition sequence, shower, and box; columns progress through successive resets.
  • Closed-loop control: State feedback lets identical skills use different controls across pattern-specific states, avoiding separate learning from scratch for every pattern.Without state feedback, an open-loop controller cannot distinguish the required inputs for identical skills in different states.
  • Regularization: Figure 4 evaluates cascade learning curves across prior-regularization coefficients using five independent trials per coefficient and mean ± standard deviation.The figure isolates how γ changes performance trends.

D. Learning with Extreme Precision

Shower and box require extreme throwing precision because short side-throw flight times prevent reliable visual tracking; adaptation improves accuracy but not deterministic convergence.

  • Short side-throw flight times fall below the robot’s sensorimotor and visual-processing latency, making reliable in-flight tracking impossible.The resulting constraint demands higher throwing precision for shower and box than for the other implemented patterns.
  • Landing distributions converge to concentrated target zones, but shower and box retain lower overall success probabilities and lack deterministic zero-failure convergence across trials.The authors suggest the purely open-loop catching policy may limit robustness for these complex patterns.

E. Safety

The Mutually Reachable Set constrains planner states to preserve safe transitions between juggling actions, preventing unreachable or non-viable commands during on-robot learning.

  • MRS motivation: The planner imposes the MRS because instantaneous joint-state bounds alone do not ensure reachability from the preceding state or viability for the next motion.The constraint prevents states that could lead to joint-limit violations in subsequent actions.
  • Overall safety: Across 7,578 planner queries, 89.0% of unconstrained solutions were unsafe, whereas all MRS-constrained solutions were reachable and viable by construction.Unsafe solutions were unreachable, non-viable, or both.
  • Pattern-specific failure modes: Without MRS, 99.6% of outside-set cascade solutions and 100% of outside-set tennis–half-shower–cascade and box solutions were unsafe.The shower pattern was an exception: 47.3% of its outside-set solutions remained reachable and viable.
  • Planner behavior: MRS-constrained solutions remain inside precomputed polytopes, while unconstrained planning selects both reachable-but-non-viable and viable-but-unreachable states.Many constrained solutions lie near polytope boundaries, indicating that the constraints are frequently active.
  • Evaluation: Table 3 compares constrained and retrospectively unconstrained planner solutions, classifying outside-MRS states as safe only when both reachability and viability hold.Because MRS is a conservative inner approximation, outside-set states are not necessarily unsafe.

3. DISCUSSION

The framework combines imperfect prior knowledge, physical experience, and safety constraints to adapt dynamic juggling skills. Its broader scope is shaped by challenges in feedback latency, low-level behavior design, skill decomposition, and forceful interactions.

  • Learning from physical experience: The robot uses an imperfect prior to guide learning where experience is limited, then increasingly relies on locally accumulated experience.This allows the system to adapt diverse juggling behaviors without learning entirely from scratch.
  • Learning from physical experience: Five three-ball juggling patterns were acquired through rapid on-robot learning, while multi-fingered contacts remained sensitive to friction, compliance, impact, and release timing.Specialized end-effectors can make interactions more predictable, whereas multi-fingered hands support more varied patterns but require greater adaptation.
  • Continual adaptation: Changes such as dust, humidity, and cleaning altered contact conditions enough to affect previously learned behaviors, motivating continued adaptation during extended operation.These physical changes were observed but not systematically evaluated.
  • Continual adaptation: Feedback latency limited the shower and box patterns because short side-throw flight times prevented visual correction before catching.The proposed direction combines task-level learning with high-bandwidth proprioceptive or tactile feedback and compliant actuation.
  • Safety: The mutually reachable set keeps constrained transition states within regions supporting safe composition, avoiding states that are reachable but not viable or viable but not reachable.The figure’s blue constrained points remain inside the precomputed polytopes, while unconstrained solutions can fall into multiple failure categories.
  • Low-level behavior design: Low-level strategies and parameterizations affect how repeatably and quickly task-level learning can improve behavior.In the box pattern, finger recontact during release made one throw more variable and the command-to-outcome relationship highly nonlinear or discontinuous.
  • Scope and future directions: A proposed extension to club juggling would pair a goal-conditioned reinforcement-learning policy for low-level behavior with the existing high-level memory-based learner.Club juggling requires coordinated control of position and orientation through finger and thumb interactions.
  • Scope and future directions: The framework’s current skill-level adaptation may not directly extend to tasks lacking clear subgoals or requiring continuous adaptation across longer skill sequences.Forceful interactions also require safety constraints that account for contact consequences beyond the robot’s kinodynamic limits.

4. MATERIALS AND METHODS

The framework is organized as three interconnected modules with distinct timing, update rates, and abstraction levels for rapid, safe adaptation of juggling skills.

  • Framework structure: The framework combines a parameterized behavior generator, a high-level optimization-based task planner, and a low-level trajectory component for physical juggling.These modules support dynamic composition of throws and catches on the robot.

A. Safe Dynamic Composition of Task-Parameterized Trajecto-

The method safely composes online-replanned throwing and catching trajectories by constraining transition states to a mutually reachable set. Offline approximation and separate timing checks preserve feasibility while exposing remaining schedule limitations.

  • Trajectory composition: Each throwing or catching skill connects commanded joint position, velocity, and acceleration states through a trajectory generated from the learner’s task-level command.Transition states are references sent to the tracking controller rather than measured robot states.
  • Trajectory composition: Including acceleration in transition states enforces continuity across consecutive trajectories and accounts for higher-order actuator dynamics and limited actuator bandwidth.The learner replans target transition states online rather than following a fixed open-loop sequence.
  • Trajectory generation: The trajectory generator connects initial and target transition states using a minimum-duration jerk-constrained trajectory under robot physical limits.Ruckig preserves the scheduled duration when feasible and otherwise returns the shortest feasible duration.
  • Mutually Reachable Set: The Mutually Reachable Set contains states for which every pair can be connected in either direction by a feasible trajectory.This bidirectional reachability supports safe composition of arbitrary successive transitions within the set.
  • Mutually Reachable Set: Because exact construction is generally intractable, the method builds a finite-dimensional inner approximation using B-spline trajectories and intersected forward- and backward-reachable sets.The resulting set is represented as a convex polytope, and every retained state satisfies the mutual-reachability condition.
  • Implementation flexibility: The offline set construction uses a trajectory parameterization distinct from the online Ruckig generator, and the low-level behavior representation could be replaced if safe composition remains assured.This separates the safety-set construction from the online trajectory-generation implementation.
  • Online safety constraint: The planner imposes the precomputed set through a linear inequality constraint when mapping learner commands to target transition states.This prevents online updates from selecting states from which subsequent skills cannot be feasibly composed.
  • Online safety constraint: Membership in the set guarantees a feasible trajectory but not completion within the orchestrator’s scheduled duration T.A transition taking longer than T can delay a catch and lead to a missed ball.

B. Regularized Memory-Based Learning for Rapid Adaptation

The method combines retrieved physical experience with a prior model to correct task-level commands online, using local fitting where data are available and prior guidance where experience is sparse. This supports stable adaptation despite inaccurate nominal predictions.

  • The learner retrieves relevant physical experiences, fits a local model regularized toward the prior, and computes a corrected task-level command for the desired outcome.When relevant experience is unavailable, the local model reduces to the prior; as experience accumulates, physical observations increasingly determine behavior.
  • The nominal throwing planner assumes ideal release and ballistic flight, but physical contact, friction, alignment, and slippage can substantially change the observed outcome.
  • The prior model remains useful even when its absolute predictions are inaccurate because its first-order gradient provides local information about state- and command-dependent outcome changes.
  • The Nadaraya–Watson memory estimate supports interpolation but becomes unreliable when relevant experience is sparse, so the prior guides extrapolation during early adaptation.
  • The local regression balances physical experience against the prior through regularization, allowing the fitted relationship to depart from the prior as observations accumulate.
  • A regularized command update avoids excessively large changes caused by imperfect local inversion and keeps commands near the locally weighted reference.The adaptation penalty uses η = 0.3 in all experiments.
  • Memory retrieval and local model fitting make the resulting policy closed-loop because the current state determines which experiences influence the command.

C. Perception

The perception system combines prompt-driven segmentation, depth-based point-cloud processing, and CAD-based tracking to estimate object motion with low latency. It predicts airborne ball states for catching and can adapt to new object geometries without retraining the perception network.

  • The perception pipeline processes synchronized RGB-D data with prompt-driven segmentation and CAD-based geometric tracking, avoiding task-specific data collection or retraining for new objects.
  • EfficientTAM provides flexible instance segmentation and runs in 0.01 second on an NVIDIA RTX A6000 GPU after initialization with four calibration images.
  • The system projects depth points into the RGB image, retains points inside object masks, and registers the resulting cloud against a CAD model to estimate rigid-body translation.
  • A rolling buffer of 3D observations is fit to a ballistic parabola, whose forward prediction becomes the target catching state passed to the task planner.
  • The same pipeline can extend to non-spherical objects by using corresponding CAD geometry to estimate position and orientation, without retraining the perception network.

This PDF file includes:

The supplementary material documents juggling targets, skill sequences, task-level planning, and the assumptions and guarantees underlying the mutually reachable set.

  • Supplementary materials: Supplementary figures and tables specify shared target positions, skill sequences, and optimization formulations for the juggling patterns.The materials include target-position coordinates, task-planner formulations, and configurations for cascade, tennis, half-shower, shower, and box patterns.
  • Skill representation: Each juggling skill is represented by a type, object ID, task identifier, and duration that coordinate task objectives and asynchronous perception.Throw and catch identifiers encode target and source positions, while object identity and duration support tracking requests after throws.
  • Task-level planning: The task-level planner solves constrained throwing and catching optimizations while requiring target joint states to remain within the mutually reachable set.Throw planning determines release conditions from landing position and flight duration; catch planning specifies interception position, orientation, and terminal rest.
  • Model assumptions: The planner assumes instantaneous ball release followed by ballistic flight, while physical contact, friction, slippage, actuation, and tracking errors create deviations from that idealization.The release acceleration constraint matches hand acceleration to gravity at the release instant to reduce relative motion at the release boundary.
  • Reachability and limitations: The mutually reachable set guarantees bidirectional connections among retained states under physical limits but is only an inner approximation based on finite-dimensional B-spline trajectories.Its polytope can be imposed directly as a linear constraint, though feasible trajectories outside the parameterized family may be excluded.
Loading 2608.26800v1…