Source-linked AI summary
Locomotion-Grounded Humanoid Soccer: Task-Gated Reinforcement Learning of a Multi-Directional Kicking Library
Abu Hanif Muhammad Syarubany, Jaehyun Jang, Hwanhee Kim, Kyuwon Kim, Seungyeon Ryu, Chang D. Yoo
TL;DR
Prior humanoid soccer systems make motion tracking the substrate, leaving locomotion exercised under ball-approach commands rather than assessed broadly. This paper trains commandable locomotion first and layers task-gated kicks on top; its seven-skill library averages 15.4% success across seven bearings and transfers to hardware.
Problem
Prior systems derive locomotion from motion tracking and exercise it under ball-approach commands, while soccer also demands general locomotion and precise striking.
Source: §I · p. 1Method
The paper trains general command-conditioned locomotion first, then adds motion-guided kicking skills as task-gated layers that return to the shared locomotion state.
Source: §I · p. 1, A. Stage 1: locomotion substrate · p. 4, B. Conclusion · p. 8Results
The seven-skill library averages 15.4% success across seven bearings, costs at most 0.04 in command-tracking MAE, and transfers to hardware.
Source: B. Conclusion · p. 8Takeaways & Limitations
The shared locomotion state serves as a composition hub, reducing transitions for N skills to O(N) rather than O(N^2), with locomotion evaluated alongside the attached library.
Source: §I · p. 2, B. Conclusion · p. 8Mean success at 0.5 m falls from 33.2% across teachers to 15.4% merged, and whether this reflects capacity or distillation remains open without seed replicates.
Source: A. Limitations · p. 8
Abstract
from arXiv · showhide
Recent humanoid soccer systems make motion tracking the substrate and derive locomotion from it, typically by steering a motion-reference anchor toward the ball. This yields strong shooting results, but locomotion is trained only on the narrow, deterministic command distribution ball approach induces, never evaluated as a capability in its own right. We invert the stack: a general, command-conditioned locomotion policy is trained first as the substrate, and N motion-guided kicking skills are added on top as task-gated layers, so the reachable gait space is set by the locomotion curriculum rather than any reference clip. Because every skill starts from and returns to this same commandable state, locomotion also becomes a composition hub (O(N) transitions rather than O(N^2)), and post-strike stabilisation is handed back to the trained controller rather than scripted per clip. We instantiate this on a 29-DoF Unitree G1 with seven retargeted kicking skills spanning 259.5 degrees of nominal aim direction, including lateral, rearward and weak-foot strikes a single forward-facing reference cannot express, and report shooting accuracy alongside command-tracking, terrain and push-recovery results with the full skill library attached, an axis prior humanoid soccer systems do not report. The library is validated on hardware across forward, lateral, rearward and commanded approaches.
I. INTRODUCTION
The paper reverses the usual tracking-first stack by training commandable locomotion as the substrate and adding task-gated kicking skills. It presents a multi-directional library and evaluates locomotion alongside shooting.
I. INTRODUCTION
- Prior systems use motion tracking as the substrate and derive locomotion by steering a tracking anchor toward the ball.This approach exercises locomotion only under ball-approach commands, rather than measuring it as a general capability.Source: §I · p. 1, Figure 1 · p. 1
- A command-conditioned locomotion policy is trained first, then motion-guided kicks are added as task-gated layers that are hard-zeroed in locomotion mode.Each skill returns post-strike recovery to the shared trained controller rather than a synthetic stabilization tail.Source: §I · p. 1
- The policy is instantiated on a 29-DoF Unitree G1 and evaluated in simulation and on hardware.Training uses IsaacLab; evaluation includes MuJoCo sim-to-sim on the exported policy.Source: §I · p. 1
- Attaching the skill library costs at most 0.04 in command-tracking MAE, and the evaluation combines shooting with locomotion measures not previously reported together.The reported axes include command tracking, terrain traversal, and push recovery with the library attached.Source: §I · pp. 1–2
- The seven-skill library spans 259.5° of nominal aim, including lateral, rearward, and weak-foot strikes.The contribution reports per-skill coverage and a union with gaps included.Source: §I · p. 2
II. RELATED WORK
Prior humanoid soccer work spans single-strike and multi-skill systems, while motion tracking supplies coordination and velocity-commanded locomotion supplies a general substrate. The paper frames locomotion as a mode in one task-conditioned policy and uses it as a shared hub for skill transitions.
A. Humanoid Soccer and Robot Sports
- Miniature-platform soccer learned walking, turning, kicking, getting up, and transitions, while full-size systems cover varied shooting and interaction tasks; no prior system earns a full mark for free command, omnidirectional locomotion, or rearward targets.SkillX composes trapping, dribbling, and shooting; RoboNaldo reports strong shooting using a three-stage curriculum.Source: A. Humanoid Soccer and Robot Sports · p. 2
B. Motion Tracking, Locomotion and Object Interaction as Primitives
- Reference-guided tracking supplies whole-body coordination but constrains timing and contact selection, while velocity-commanded locomotion with terrain and disturbance curricula is established as a separate capability.The paper places that locomotion capability at the base of the stack rather than treating it as a ball-approach by-product.Source: B. Motion Tracking, Locomotion and… · p. 2
C. Skill Libraries, Composition and Transitions
- A shared locomotion hub makes each skill’s termination state another skill’s initiation state by construction, reducing transitions from O(N^2) to O(N).Specialists are merged by behavior distillation, with a student regressing onto routed teacher actions without a reward term in merging.Source: C. Skill Libraries, Composition and… · p. 2
III. PROBLEM FORMULATION
- The formulation uses one policy for a task-conditioned MDP, with task mode selecting the reward and termination specification at each timestep.LOCO is an ordinary mode in the mode set, not a residual of ball approach.Source: §III · p. 2
A. Task-mode gating
Task mode gates reward terms inside a single policy: each term is active only in its authored mode, and kick objectives are zeroed rather than discounted during locomotion. The observation retains inactive-mode slots while indicating the current mode.
A. Task-mode gating
- Backward MAE rises 0.12 at a 0.1× tracking floor and 0.33 at 0.5×; all four command axes degrade monotonically when tracking is discounted instead of zeroed.The 0.5× setting is worse at all eight matched checkpoints (sign test p=0.008).Source: A. Task-mode gating · p. 2
- The mode gate is a hard zero: in locomotion mode, kick terms contribute nothing to the gradient, and the reverse applies outside their authored mode.Reward, termination, and observation are gated inside one network rather than handled by arbitrating separate controllers.Source: A. Task-mode gating · p. 2
- Inactive-mode observation terms are masked to zero but remain in the fixed 261-dimensional input, while the locomotion/kick one-hot remains visible.The mode indicator tells the policy its mode rather than requiring it to infer mode from zeroed observation blocks.Source: A. Task-mode gating · pp. 2–3
B. Skill conditioning
Kicking skills are conditioned on retargeted reference trajectories and a bounded relative aim command. When a reference ends, control returns to the shared locomotion policy, which receives the post-strike state without a synthetic recovery reference.
B. Skill conditioning
- Each skill uses its retargeted reference trajectory as an observation, making the policy reference-conditioned rather than index-conditioned.The strike direction is commanded rather than specified as a world-frame target point.Source: B. Skill conditioning · p. 3
- The target is set five metres from the ball along the skill’s nominal bearing plus commanded offset, and the policy receives the normalized offset bounded by construction.The fixed 45° reference angle means the command needs no robot localization at deployment.Source: B. Skill conditioning · p. 3
C. Handoff
- When authored reference content ends, the mode returns to LOCO and the trained locomotion policy resumes control without tracking a synthetic recovery reference.The controller takes over in the single-support, momentum-carrying state left by the strike; kick entry is commanded, and entry accuracy falls with carried velocity.Source: C. Handoff · pp. 3–4
- The actor observation has 261 dimensions; its terms are robot-measurable except base linear velocity, which is critic-only.The observation-space description accompanies the deployed architecture.Source: C. Handoff · p. 4, Table II · p. 4
IV. METHOD
Training proceeds from a command-conditioned locomotion substrate to motion tracking and then motion-guided kicking, preserving locomotion as the base task while adding strike objectives.
A. Stage 1: locomotion substrate
- The Stage-1 command curriculum alone sets the reachable gait space, using forward, backward, lateral, and in-place-rotation commands over mixed terrain with disturbances.No ball or reference clip is present during this stage.Source: A. Stage 1: locomotion substrate · p. 4
B. Stage 2: motion tracking
- Kick-partition environments add human-clip tracking while ball and target rewards remain zero; locomotion environments continue the unmodified Stage-1 task.The clip is recovered from video and retargeted to the robot.Source: B. Stage 2: motion tracking · p. 4
C. Stage 3: motion-guided ball kicking
- The shooting objective ramps linearly from zero over 5,000 iterations and stays at zero throughout Stage 2; each attempt samples aim within ±15° of the skill’s nominal bearing.Ball placement is randomized by ±10 cm, and the alive reward is attenuated 10× before contact.Source: C. Stage 3: motion-guided ball kicking · p. 4
- Balance-potential shaping supplies a signal where tracking terms saturate near falling; with the stated discount and termination conditions, it leaves the optimal policy unchanged.The potential combines capture-point distance to the support polygon with a fall-height margin.Source: C. Stage 3: motion-guided ball kicking · p. 4
- Locomotion environments are seeded from kick-disturbed poses, more often from live kick states than reference poses, to train the handoff boundary.This seeding is enabled during distillation; a specialist-level retraining ablation isolates its effect.Source: C. Stage 3: motion-guided ball kicking · p. 4
D. Stage 4: specialist distillation
Locomotion-grounded specialists are consolidated into a single deployable actor through DAgger-style distillation using the student’s own rollout distribution.
D. Stage 4: specialist distillation
- One locomotion-grounded specialist is trained per clip, then the specialists are merged into one deployable actor by DAgger-style distillation.The teachers label student states selected by a skill-index router.Source: D. Stage 4: specialist distillation · pp. 4–5
- Distillation supervision follows the student’s induced state distribution rather than teacher trajectories, covering states the student drifts into.Kick-partitioned environments retain their specialist through locomotion ticks; locomotion-only environments sample a specialist uniformly.Source: D. Stage 4: specialist distillation · p. 5
- Each teacher uses its own frozen observation normalizer, and the rollout action form must match the regression target.The passage warns that mismatches can produce confident wrong actions or self-amplifying regression.Source: D. Stage 4: specialist distillation · p. 5
E. Learning algorithm
The learning setup uses mode-specific algorithm targets for locomotion and kicking, with domain randomization broadened at the transition into motion tracking.
E. Learning algorithm
- FastSAC is used because replay extracts more updates per simulated step when simulation throughput is the bottleneck in contact-rich training.It is an off-policy maximum-entropy actor-critic.Source: E. Learning algorithm · p. 5
- The discount and entropy target depend on the active mode, so the kick’s multi-second horizon is not imposed on shorter-horizon locomotion.The mode-dependent targets are γ(mt) and α(mt).Source: E. Learning algorithm · p. 5
F. Domain randomisation
- Randomization broadens at the Stage-1-to-Stage-2 boundary and then remains constant, including contact-solver response and strike-specific joint armature and friction.Ball observations also receive noise, episode bias, delays, held samples, and stale-reading probability.Source: F. Domain randomisation · p. 5
A. Platform, simulation and skill library
The study evaluates a seven-skill Unitree G1 policy in simulation and on hardware, reporting shooting quality alongside locomotion generality with the library attached.
A. Platform, simulation and skill library
- Seven skills are retargeted from separate human clips, with each reachable aim band extending ±15° around its configured nominal bearing.Nominal bearing is determined by configured ball and target placement.Source: A. Platform, simulation and skill… · p. 5
B. Metrics and protocol
- Shooting is scored by target-distance success at 0.5 m and 1.0 m radii, alongside shot error, peak ball speed, and contact rate; locomotion measures include tracking, terrain, and push recovery.All locomotion measures are evaluated with the library attached.Source: B. Metrics and protocol · p. 5
- Unified-policy results are medians across checkpoint sweeps of 8 checkpoints, or 16 for N=7, with 250–300 attempts per checkpoint.Single-checkpoint evaluation is described as unreliable; one skill ranged from 3.6% to 45.2% across a 16-checkpoint sweep.Source: B. Metrics and protocol · p. 5
A. Shooting quality
- 44.3% success at 0.5 m is reported for the skill-1 specialist, but the distilled skill-1 policy falls to 10.2% and the library mean is 15.4%.The specialist result uses a ±15° aim cone, versus ±38.7° for the comparison, and roughly a quarter of its ball speed; the study cautions this is not a like-for-like win.Source: A. Shooting quality · pp. 5–6
- The library is tuned for directional coverage at a survivable strike, while peak ball speed is the column where prior work dominates.In five teachers with logs, shooting receives 10–15% of positive kick-mode reward and tracking receives 76–85%; raising shooting weight several-fold previously collapsed sim-to-sim survival.Source: A. Shooting quality · p. 6
- Hardware tests cover forward, lateral, and rearward skill groups plus commanded approaches, with success radius scaled to the shorter 1.42–2.08 m test-room distance.The radius is scaled to 0.14 m so trials retain the simulated ±5.7° tolerance; Fig. 3 depicts commanded locomotion, strikes, and the kick-to-locomotion handoff.Source: A. Shooting quality · p. 6, Figure 3 · p. 6
B. Kicking skill coverage
The seven-skill unified policy spans separated contact regions, but placement robustness is stronger for contact and stability than for aiming. Mean success declines as the library grows.
B. Kicking skill coverage
- 564 of 700 attempts made contact across seven skills; contacts form five islands, with skills 1/6 and 5/7 overlapping.Post-contact ball speed ranges from 2.04 to 4.13 m/s.Source: B. Kicking skill coverage · p. 6, Table VI · p. 6, Figure 4 · p. 7
- Across 1,500 placements, contact was 70.4%; 78.5% lay outside every skill’s training box, and only 11.9% passed within 1.0 m of the target.Mean topple rate was 0.6%, with a 2.6% worst case across the seven bearings.Source: B. Kicking skill coverage · p. 6
- Mean success fell from 26.0% at N=2 to 15.4% at N=7, with the deficit present at the smallest library and widening as skills were added.The authors identify interference from merging skills as the dominant cause.Source: B. Kicking skill coverage · p. 6, Table VII · p. 6
- Skill 2 contacted the ball in 92.0% of attempts but had 0.8% median success, illustrating that contact did not ensure accurate aiming.Skills 1 and 6 contacted on every attempt but landed only 10.2% and 7.6% of the time, respectively.Source: B. Kicking skill coverage · p. 6
C. Handoff
The policy survives kick-to-locomotion returns and mid-swing aborts, while moving entries trade accuracy and stability for speed. The evaluation also identifies limits in merged-skill accuracy and in the evidence that attaching skills preserves locomotion performance.
C. Handoff
- At full-speed kick entries, 95.8% survived versus 99.3% from rest, while success at 0.5 m fell from 17.0% to 11.9%.Across the tested entry-speed envelope, contact fell from 95.3% to 84.1% and topple rate rose from 0.7% to 4.2%.Source: C. Handoff · p. 7, Table VIII · p. 7
- Live kick-state recovery training and post-strike stabilisation address two distinct robustness mechanisms; the paper also reports hardware survival under manually triggered mid-swing aborts.The stabilisation ablation’s arm alone settled, while live-donor performance differed from reference-clip training by 6.6 to 39.7 points across the tested arms.Source: C. Handoff · p. 7, Table X · p. 7, Table IX · p. 7
D. Locomotion generality
- All tracking deltas were non-negative but within one standard deviation of the baseline’s three-seed spread, so small reproducible degradation could not be excluded.Terrain traversal used the training criterion of 10 consecutive ticks below threshold, rather than an instantaneous dip.Source: D. Locomotion generality · p. 7, Table XI · p. 8
A. Limitations
- Mean success at 0.5 m fell from 33.2% across teachers to 15.4% merged; whether this reflects capacity or distillation artifacts remains open.Distillation had no seed replicates, and the multi-skill claim is therefore about coverage and stability, not specialist accuracy.Source: A. Limitations · pp. 7–8
- Correcting each skill’s bearing improved success at both radii by 1.7–1.8× on the same trials, but the offset fit must be made per deployed checkpoint.Offsets replicated across halves of a checkpoint but not across checkpoints.Source: A. Limitations · p. 8, Table XII · p. 8
B. Conclusion
The policy trains commandable locomotion first, then adds task-gated kicking skills. Its seven-bearing library reports shooting results, a bounded command-tracking cost, and hardware transfer, while shared locomotion provides a hub for skill composition.
B. Conclusion
- Skills are hard-zeroed outside kick mode, fixing reachable gait space to the command curriculum; their shared locomotion start and end state yields O(N), not O(N^2), transitions.Source: B. Conclusion · p. 8
- 44.3% success at 0.5 m for the specialist uses a narrower aim cone and lower pace than prior work; the distilled library averages 15.4% across seven bearings.Source: B. Conclusion · p. 8
- The distilled library transfers to hardware at a locomotion cost of at most 0.04 in command-tracking MAE.Source: B. Conclusion · p. 8