PaperlayerView exact revision on arXiv

Source-linked AI summary

Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skills

Gaytri Jena, Kapil Wanaskar, Vinija Jain, Aman Chadha, Vasu Sharma, Amitava Das

arXiv:2608.01851v1cs.ROcs.AI

TL;DR

Robot learning is organized around whether competence is shipped as frozen weights or written and refined as executable skills. This survey builds a taxonomy around that fault line and finds that the open-ended loop combining feedback, memory, and search remains sparsely populated.

  • Problem

    Robot learning lacks a unified account of the weights-versus-skills divide and the degree of self-improvement among code-as-policy systems.

  • Method

    The survey positions 77 systems across six branches and operationally arranges code-as-policy methods by feedback, memory, and search.

  • Results

    The combined feedback-memory-search cell is the least populated, containing only a few recent systems including ASPIRE [160], ENPIRE [272], and RoboClaw [139].

  • Takeaways & Limitations

    Downloaded robot skills remain static replays, making adaptation to a target robot a central open challenge for skill marketplaces.

  • Takeaways & Limitations

    The survey is deliberately code-centric, reports capabilities qualitatively rather than ranking methods, and covers a rapidly changing field.

Abstract

from arXiv · show

Robot learning is splitting into two bets: policies that bake competence into frozen weights (vision-language-action, or VLA, models), and agents that write and refine their own executable skills as code. This survey organises the field around that axis of weights versus skills. Its central analytical contribution is a deep-dive that arranges code-as-policy methods by their degree of self-improvement, from zero-shot program synthesis, through closed-loop self-repair and persistent skill memory, to the sparsely populated cell in which execution feedback, skill memory, and evolutionary search combine into one open-ended loop; only a few very recent systems (for example ASPIRE, ENPIRE, and RoboClaw) occupy that cell. We map the complementary "skills" pole, from unsupervised reinforcement-learning skill discovery to large-language-model skill libraries, and show that the word "skill" is used in at least five distinct senses, of which only the code sense self-improves without gradient updates. We then connect the taxonomy to the emerging skill economy: commercial robot-skill marketplaces now distribute one-tap skills across robots but ship only static playback, which surfaces open problems of adaptation, cross-embodiment portability, provenance, safety verification, composition, and standardisation. This is a deliberately focused survey. Rather than cataloguing the field exhaustively, it examines 77 representative systems across six technique families through one taxonomy and a set of contrast tables, and it supplies operational definitions of the self-improvement mechanisms together with a statement of what each family cannot do.

1 INTRODUCTION

This focused survey organizes robot learning around frozen-weight VLA policies versus executable code-as-policy agents that improve from experience [144] [160]. It examines 77 systems through a single taxonomy while connecting skill creation, distribution, and adaptation to the emerging robot-skill economy.

  • Organizing question: Robot learning is organized around frozen-weight VLA policies that map observations to actions versus code-as-policy agents that write executable programs and improve from experience [144] [160].Figure 1 presents these as the survey’s two organizing poles.
  • Skill economy: Its skill-stack framing separates Build, Distribute, and Adapt, while Unitree UniStore distributes one-tap, cross-model skills that remain static playback.The promised adaptation example—adjusting grip per fruit to prevent bruising—is identified as beyond static playback.
  • Scope and contributions: The survey examines 77 selected systems in one analytical taxonomy, excluding pure perception, navigation, and locomotion except when directly relevant to authoring and improving manipulation skills.It is explicitly a focused survey rather than an exhaustive census.
  • Related surveys and taxonomy: The survey’s distinctive contribution is organizing code-as-policy methods by degree of self-improvement, including a dedicated cell absent from prior surveys.Table 1 contrasts coverage of weights, code, skills, reward, self-improvement, and the robot skill marketplace; Figure 3 places related surveys on a timeline.

2 A TAXONOMY OF ROBOT-LEARNING TECHNIQUES

The taxonomy divides robot-learning techniques by what ships—frozen weights or executable skills—and tags systems by Feedback, Search, and Memory. It organises six branches and compares representative systems by shipped artifact, evaluation setting, and self-improvement from experience.

  • Taxonomy overview: The field is organised around six branches split by whether systems ship frozen weights or executable skills, with Feedback, Search, and Memory as recurring mechanism tags.The taxonomy covers code-as-policy, end-to-end VLA policies, LLM-authored rewards or curricula, robot skill libraries, sim-to-real and cross-embodiment transfer, and benchmarks or simulators.
  • Comparison framework: Table 2 compares representative systems by what they ship, evaluation domain, and whether they improve from their own experience.Its detailed comparison covers 47 of the 77 taxonomy systems, while the code-as-policy and skill-library branches are tabulated fully in Tables 4 and 6.
  • §3.1 Code-as-policy for robots: Code-as-policy systems span zero-self-improvement methods such as Code-as-Policies [144], ProgPrompt [236], and VoxPoser [100] to self-improving systems including CaP-X [58], RoboClaw [139], ASPIRE [160], and ENPIRE [272].The listed systems are evaluated in simulation or on real robots; self-improvement is marked ✘ for the earlier examples and ✔ for the latter systems.
  • §3.2–§3.5 Comparative branches: VLA and generalist-policy systems ship weights and are listed without self-improvement, whereas reward, skill-library, and transfer families include systems marked ✔ or ~ for experience-based improvement.The taxonomy lists seven reward or curriculum systems, seven skill-library or lifelong-learning systems, and four sim-to-real or cross-embodiment systems with their evaluation settings.
  • Corpus scope: The taxonomy corpus contains 77 systems selected through seed-and-snowball curation, while a separate landscape corpus contains 225 works gathered through structured searches.Taxonomy placement requires passing tests for axis placement, while corpus inclusion separately requires a relevant embodied-agent focus and verifiable metadata.

3 THE TECHNIQUE FAMILIES

The survey examines six technique families, centering on code-as-policy as the only branch organised by degree of self-improvement. Its ladder tracks whether systems generate executable code, use execution feedback, retain validated code, and search across programs, with the highest combined regime sparsely populated.

  • Overview: The taxonomy covers six branches: code-as-policy, weights-based, reward-synthesis, skill-library, transfer, and benchmark methods.
  • Overview: Code-as-policy is the survey’s focal family and the only branch organised by how much competence systems produce at run time, rather than by application or embodiment.
  • Code-as-policy: The code-as-policy ladder asks whether systems write control code, repair it from within-task execution feedback, remember validated code across tasks, search program populations, and combine these mechanisms in an open-ended loop.
  • Code-as-policy: Only a handful of the thirteen zero-shot systems reach the combined feedback, memory, and search regime, motivating explicit operational definitions for these mechanisms.
  • Code-as-policy: The lowest rung contains systems that emit executable control code without closing an execution loop back to generation, using language models as synthesizers over fixed perception and control primitives.

CogACT

CogACT organizes code-as-policy robot-learning methods by the self-improvement mechanisms they combine: feedback, persistent memory, and population-based search. Its highest rung is a full open-ended loop combining all three, occupied by only ASPIRE [160], ENPIRE [272], and RoboClaw [139].

  • Closed-loop self-repair: Closed-loop self-repair revises a single program from execution-grounded feedback, but corrections are discarded at task boundaries and do not accumulate across tasks.Inner Monologue [102], DoReMi [73], REFLECT [159], Code-as-Monitor [306], AHA [48], and Introspective Planning [145] occupy this feedback-only rung.
  • Skill-library accumulation: Skill-library accumulation makes validated code persistent across tasks, allowing later tasks to retrieve and compose a growing library, as in Voyager [250] and related systems.This memory rung is catalogued as a separate technique family in §3.4.3 and includes Voyager [250], LRLL [242], Uni-Skill [274], RoboCoder [137], and DROC [294].
  • Evolutionary program search: Evolutionary program search maintains multiple candidate programs, scores them by execution, and mutates the best rather than repeatedly repairing one candidate.CaP-X [58], RoboEvolve [26], and Code Evolution (GEAR) [72] occupy the feedback-plus-search rung without persistent memory.
  • Full self-improving loop: Only ASPIRE [160], ENPIRE [272], and RoboClaw [139] combine execution feedback, persistent skill memory, and population-based program search in one full self-improving loop.ASPIRE uses multimodal execution traces, validated code, and population search; ENPIRE adds automatic reset, parallel rollouts, and log-driven revision.
  • Taxonomy overview: Table 4 classifies 30 code-as-policy systems by feedback, memory, and search, spanning zero-shot synthesis with none of these mechanisms through partial rungs and the full loop.Zero-shot systems such as Code-as-Policies [144], ProgPrompt [236], VoxPoser [100], and Instruct2Act [97] emit executable code but provide no feedback, memory, or search.

3.2 End-to-End Vision-Language-Action Models · 3.3 Reward and Curriculum Synthesis

End-to-end VLA models place robot competence in frozen weights that map pixels and language directly to actions, whereas reward- and curriculum-synthesis methods use language models within reinforcement-learning loops but ultimately still ship learned weights. The former lack inspectable self-improvement loops, while the latter can evolve rewards, environments, and curricula, including transfer to real robots.

  • 3.2 End-to-End Vision-Language-Action Models: VLA models regress motor actions directly from pixels and language through a vision-language backbone plus a pluggable action head, storing competence in frozen weights rather than inspectable programs [15] [120].This family includes systems ranging from RT-1 and RT-2 to open reproductions such as Octo and OpenVLA [17] [18] [66] [120].
  • 3.2 End-to-End Vision-Language-Action Models: VLA action representations have progressed from coarse discrete token heads toward continuous flow- and diffusion-based heads for smoother, higher-frequency control [15] [14] [138] [207].Examples include π0’s flow-matching head, π0.5’s open-world extension, CogACT’s cognition/action-module separation, and SpatialVLA’s 3D spatial encodings.
  • 3.2 End-to-End Vision-Language-Action Models: Large foundation-model efforts extend the VLA paradigm to humanoid platforms and embodied reasoning, including GR00T N1’s dual-rate reasoner/action architecture and Gemini Robotics [194] [65].GR00T N1 pairs a slow System-2 vision-language reasoner with a fast System-1 diffusion transformer for high-rate motor actions.
  • 3.2 End-to-End Vision-Language-Action Models: Unlike self-improving code systems, VLAs have no execution loop or code: they acquire competence through scaling data and parameters, generalise by interpolation rather than search, and cannot be edited, audited, or recombined after training.This limitation is the survey’s reason for treating end-to-end VLAs as the foil to code-based policies.
  • 3.3 Reward and Curriculum Synthesis: Reward- and curriculum-synthesis methods keep the language model as programmer, but have it write rewards, environments, or curricula that reinforcement learning compiles into policy weights.The model reads training statistics and revises its generated artefact, yet the shipped result remains weights rather than self-improving code.
  • 3.3 Reward and Curriculum Synthesis: Eureka [163] evolves executable rewards using textual training-statistic reflections, exceeding expert human rewards on 83% of 29 IsaacGym tasks, while DrEureka [164] synthesises domain-randomisation ranges for zero-shot real-world locomotion.Eureka starts from unmodified environment source code and a task description; DrEureka extends the recipe toward sim-to-real transfer.
  • 3.3 Reward and Curriculum Synthesis: Generative synthesis also creates progressively harder curricula, tasks, scenes, and supervision, enabling parkour transfer to a real robot and effectively unlimited simulated training data [146] [256].Eurekaverse [146] proposes increasingly difficult environments, RoboGen [256] runs a propose-generate-learn cycle, and Auto MC-Reward [136] automates dense rewards for open-world Minecraft.

3.4 Skill Libraries: How a Repertoire is Built

Section 3.4 distinguishes skill repertoires learned as latent reinforcement-learning policies or skill spaces from retrievable code and agent libraries. It shows that only code skills self-improve without gradients, while app and market skills are the readily distributable forms.

  • 3.4 Skill Libraries: The taxonomy separates latent-RL policies, skill-space representations, retrievable code libraries, and open-world agent libraries, with only code skills self-improving without gradients.This branch asks how a repertoire is built, complementing the question of how code skills improve in §3.1.
  • 3.4.1 Unsupervised skill discovery: Unsupervised skill discovery learns diverse, distinguishable behaviors without task rewards or code, exemplified by DIAYN [50], DADS [229], LSD [200], CIC [129], and METRA [202].DIAYN [50] maximises mutual information between latent skill codes and visited states, while DADS [229] makes skills dynamics-aware for model-based control.
  • 3.4.2 Skill-based / hierarchical RL: Skill-space methods extract continuous representations from offline data: SPiRL [206] learns a skill prior, OPAL [6] discovers temporally extended primitives, PARROT [235] learns an invertible prior, and SkiMo [231] plans in skill space.These methods accelerate downstream reinforcement learning or support planning and dynamics modelling within the learned skill space.
  • 3.4 Skill Libraries: “Skill” spans at least five conflated senses, but only the code sense is simultaneously inspectable, adaptable, and composable; app and market senses are readily distributable.Table 5 records inspectability, adaptability, symbolic composability, and cross-robot distributability, exposing the mismatch that the skill economy must close.
  • 3.4.3–3.4.4 LLM and agent libraries: Code libraries store retrievable programs, while agent libraries accumulate reusable skills from open-ended experience; examples include Voyager [250], LRLL [242], BOSS [297], GITM [309], ExpeL [301], and Odyssey [157].The surveyed code systems use verified-skill curation, imitation, language-model bootstrapping, or instruction relabelling; embodied agents add text, multimodal, or open-world memories.

3.5 Sim-to-Real and Cross-Embodiment Transfer · 3.6 Benchmarks and Evaluation · 3.7 Synthesis: what each family cannot do

The survey treats sim-to-real and cross-embodiment transfer as prerequisites for useful skills, while existing benchmarks largely measure one-shot success rather than improvement through interaction. Across six evaluation axes, no robot-learning family is strong on every dimension, leaving the full-loop self-improvement cell uniquely complete.

  • 3.5 Sim-to-Real and Cross-Embodiment Transfer: Open X-Embodiment [195] pools over a million trajectories across 22 embodiments and shows positive transfer and emergent capabilities in RT-X models.The mixture combines 60 datasets from 21 institutions and hundreds of skills.
  • 3.5 Sim-to-Real and Cross-Embodiment Transfer: CrossFormer [44] trains one transformer on 900K trajectories across 20 embodiments without manually aligning observation or action spaces, matching specialist policies.The represented platforms include arms, wheeled robots, quadrotors, and quadrupeds.
  • 3.5 Sim-to-Real and Cross-Embodiment Transfer: RoboCat [16] adapts to new tasks from as few as 100 demonstrations, then generates data for the next training round in a rudimentary self-improvement loop.This adaptation directly reflects the cross-embodiment challenge faced by marketplace skills advertised for multiple robot platforms.
  • 3.6 Benchmarks and Evaluation: LIBERO [149], Meta-World [286], RLBench [105], ManiSkill2 [71], Robosuite [310], and BEHAVIOR-1K [134] provide simulated benchmarks spanning lifelong, multi-task, vision-based, and household activities.BEHAVIOR-1K specifies 1,000 everyday household activities and scores both success and efficiency.
  • 3.6 Benchmarks and Evaluation: Table 7 shows that only the full-loop cell, including ASPIRE, ENPIRE, and RoboClaw, combines a persistent skill library with feedback- and search-driven self-improvement.Every other camp lacks at least one of these axes.
  • 3.6 Benchmarks and Evaluation: Current benchmark suites mainly report one-shot competence, measuring policy success rather than improvement with experience or held-out success versus accumulated interaction.No standard benchmark yet plots the interaction-based quantity targeted by the survey’s self-improvement ladder.
  • 3.7 Synthesis: what each family cannot do: Table 8 finds that across data needs, task horizon, transfer, interpretability, safety, and failure mode, every family has a weakness and none is favourable on all six axes.This trade-off is inherited by the emerging skill economy.

4 THE SKILL ECONOMY: OPEN PROBLEMS

Commercial robot-skill marketplaces such as UniStore make adaptation, portability, provenance, safety, composition, and standardisation urgent open problems rather than hypothetical concerns. A genuine ecosystem must reconcile static downloaded packages with adaptive, trustworthy, interoperable, and jointly maintained skill libraries.

  • 4 THE SKILL ECONOMY: OPEN PROBLEMS: Commercial marketplaces now ship one-tap, cross-model motion packages, but downloaded skills are replayed without adapting to the target kitchen, gripper, or object.Within-task repair, persistent memory, and search could close this static-to-adaptive gap, turning stores of animations into stores of capabilities.
  • 4 THE SKILL ECONOMY: OPEN PROBLEMS: Cross-embodiment portability remains open because differing morphologies, action spaces, and dynamics require shared interfaces or explicit embodiment adaptation for correctness guarantees.UniStore advertises skills across G1, H1, B2, and Go2 platforms, but formal certification requirements for a new body remain unspecified [195].
  • 4 THE SKILL ECONOMY: OPEN PROBLEMS: Third-party skill marketplaces make provenance a safety property, yet no accepted standard exists for signing, attesting, or auditing a skill’s origin.Safety verification is generally undecidable for code-as-policy skills and depends on encountered scenes, motivating declared preconditions, sandboxed screening, simulation, and runtime monitors such as Code-as-Monitor [306].
  • 4 THE SKILL ECONOMY: OPEN PROBLEMS: Skill composition requires shared interfaces and pre-/post-condition calculus, but opaque motion packages do not compose and code skills compose only within one authoring system.The intended capability is composing skills such as “make coffee” and “clear the table” into a morning routine.
  • 4 THE SKILL ECONOMY: OPEN PROBLEMS: Scalable reuse needs cross-vendor skill ontologies defining preconditions, expected effects, embodiment profiles, and relations such as requires, provides, and substitutable-by.Prior work points the way, but no cross-vendor standard exists for the genuine ecosystem described here.
  • 4 THE SKILL ECONOMY: OPEN PROBLEMS: The field must reconcile self-built libraries grown from experience with marketplace libraries, while the hybrid commons model’s incentives, quality control, and feedback dynamics remain unstudied.The envisioned regime lets an agent both contribute to and draw from a shared library such as UniStore.

5 LIMITATIONS OF THIS SURVEY

The survey is deliberately code-centric: it organises systems by degree of self-improvement while treating weightbased policies and reinforcement-learning skill discovery as context rather than taxonomy subjects.

  • Scope boundaries: The organising axis is deliberately code-centric, foregrounding systems that emit and revise programs.The survey acknowledges that a representation-learning-centred map would differ.
  • Scope boundaries: Weightbased policies (§3.2) and reinforcement-learning skill discovery (§3.4) provide context, while real-world data collection is outside the survey’s central framing.A survey centred on real-world data collection would draw the map differently.

6 FUTURE DIRECTIONS

The future agenda is made actionable through four measurable quantities: autonomous improvement, compounding skill reuse, cross-embodiment transfer loss, and provenance verification. It also calls for shared evaluation protocols that measure self-improvement under a common interaction budget.

  • 6 FUTURE DIRECTIONS: Four quantities anchor the agenda: the success-vs-interactions curve, skill-library reuse rate, cross-embodiment transfer drop, and provenance check.Together, these test whether competence improves with autonomous experience, stored skills compound, skills transfer across bodies, and distributed skills retain verifiable origins and test records.
  • 6 FUTURE DIRECTIONS: The field needs standardized self-improvement evaluation using shared protocols that publish success-vs-interactions and skill-library reuse curves under a common interaction budget.Existing top-rung systems report improvement on their own splits; a benchmark modeled on LIBERO’s standardization of lifelong manipulation [149] would enable comparable assessment.
  • 6 FUTURE DIRECTIONS: Figure 11 presents a schematic protocol contrasting rising held-out success for self-improving systems with flat performance for static playback skills.Its panels also target increasing skill-library reuse and measuring success loss when transferring from a training embodiment to an unseen one; they show measurement axes and target shapes, not results.

7 CONCLUSION

The survey organizes robot learning around whether competence is shipped as weights or as skills. It maps the field into six branches and analyzes code-as-policy methods by increasing degrees of self-improvement, using end-to-end vision-language-action models as the weights-based foil.

  • The survey’s central organizing question is whether robot competence should be shipped as weights or as skills.
  • It maps robot learning into six branches and positions end-to-end vision-language-action models as the weights-based foil.
  • Its code-as-policy deep dive orders methods by self-improvement, from one-shot program synthesis through within-task repair, persistent skill memory, and population search.

A THE BROADER ROBOT-LEARNING LANDSCAPE

Table 11 broadens the survey beyond its 77-axis-selected taxonomy systems to 225 representative works across 11 robot-learning areas, comparing what methods ship, how they learn, evaluation, embodiment, and run-time self-improvement. Across these families, run-time self-improvement is uncommon, with most policies, planners, skill-discovery methods, and world-model systems marked ✘, while only a few are marked ~ or ✔.

  • Scope and comparison: Table 11 catalogs 225 representative works beyond the taxonomy’s 77 systems, grouping them into 11 areas and comparing six axes including run-time self-improvement.The axes are Ships, Learn, Eval, Embod., and run-time S.I., with self-improvement coded as ✔, ~, or ✘.
  • Imitation & diffusion policies: Imitation and diffusion policies are predominantly static policy or weight outputs, with the listed systems uniformly marked ✘ for run-time self-improvement.The family includes real- and simulation-evaluated arm and mobile systems spanning 2021–2024.
  • LLM/VLM planning & task-and-motion: LLM/VLM planning methods span simulated, real, and game settings, but their listed self-improvement status is mostly ✘ or ~ rather than ✔.The table includes task planning, reward synthesis, and generative data or augmentation families.
  • Unsupervised skill discovery & hierarchical RL: Unsupervised skill-discovery and hierarchical-RL systems are largely marked ✘, with VISR [87] shown as a partial ~ exception among the listed works.The family contains 25 representative systems using RL, self-supervision, or model-based learning across simulated, game, and multi-embodiment evaluations.
Loading 2608.01851v1…