PaperlayerView exact revision on arXiv

Source-linked AI summary

What Stops Recursive Self-Improvement in Robotics? Lessons from 123 Rounds of Agentic Skill Discovery

Jiaming Wang

arXiv:2609.31760v1cs.ROcs.AIcs.LG

TL;DR

This paper asks whether an agent can improve a robot through repeated failure diagnosis and tested changes, and examines that question in a 123-round household-manipulation run. The agent produced accepted improvements but did not solve the fridge task; the authors conclude that progress was constrained by the surrounding perception, skill-chain, and harness design rather than chiefly by the agent.

  • Problem

    The paper asks whether a coding agent can discover missing robot capabilities from failures and whether its successive changes can add up to task improvement.

    Source: §1 · p. 1

  • Method

    The system uses agents to diagnose failures, revise task programs and skills, or find and install external models, with changes tested in simulation and no human-written robot code.

    Source: §1 · p. 1, §2 · p. 2

  • Results

    After 123 rounds, the target fridge task never succeeded, and the authors identify relational perception, early-step skill-chain failures, and the evaluation harness as barriers to progress.

    Source: §2 · p. 2, §11 · p. 10

  • Takeaways & Limitations

    The authors argue that robotics self-improvement depends on representable failures, expressible fixes, and resolvable tests, which must be built into the system.

    Source: §11 · p. 10

    The study is one run on one simulator, one kitchen layout, and three tasks, without independent replicates, matched-budget baselines, or ablations.

    Source: §10 · p. 10

Abstract

from arXiv · show

Can a robot improve itself the way coding agents now improve software? We built an agentic system to find out. It watches a robot fail, works out which capability is missing, writes new skills or finds and installs external models, tests every change in simulation, and repeats, with no human writing robot code. We ran it for 123 improvement rounds on household manipulation tasks. This report describes what we learned. The good news is that the agent can discover capabilities on its own: noticing that its targets were out of view, it asked for an active-viewing model, debugged it, and deployed a working search skill. The bad news is that its improvements did not add up. Changes kept passing their tests, yet the target task, putting condiments on the top shelf of a fridge, never succeeded. We found that the agent was rarely the bottleneck. Three things around it were. First, chained perception modules do not understand relations. Segmenters such as SAM 3 find shelves but not "the top shelf", so the agent filled the gap with ever more geometric rules that never converged, when what it needed was a different kind of model. Second, skill chains lock learning onto the first step. Long tasks mostly fail early, so evidence and fixes pile up there, and later skills are rarely reached, tested, or improved. Third, what the agent learns is decided by the harness. The agent optimized exactly what the evaluator measured, including where it was wrong, and weak tests and misleading memory turned activity into a standstill. We distill these lessons into concrete recommendations for building robot systems that improve themselves, each paired with an experiment that could prove it wrong.

1 Introduction

Coding agents suggest a way to automate the diagnosis and repair of robot failures, but whether they can discover missing capabilities and make improvements accumulate remains an open question. This paper tests that idea in a 123-round household-robot experiment.

1 Introduction

  • Coding agents can read logs, form hypotheses, write and test code, and improve systems in repeated loops, motivating their use for robot self-improvement.Prior examples include growing a Minecraft skill library, evolving robot-learning reward functions, and rewriting agent code [1] [2] [3].Source: §1 · p. 1
  • The proposed loop lets an agent diagnose robot failures, add skills or find models, test changes, and repeat without a human writing robot code.The system was tested on placing condiments on a fridge’s top shelf for 123 rounds, with changes tested in simulation before use.Source: §1 · p. 1
  • The study asks whether an agent can discover missing capabilities from failures and whether successive improvements leave the robot better equipped.These questions distinguish capability discovery from cumulative task progress.Source: §1 · p. 1

3. If not, what stops them, and is it something a stronger agent

The agent sometimes discovered and deployed a missing capability, but its accepted improvements did not solve the fridge task. The paper attributes the stalled progress to relational perception, early-step concentration in skill chains, and a harness that shaped what the agent learned.

3. If not, what stops them, and is it something a stronger agent

  • The fridge task never succeeded despite accepted changes passing tests, and in the final twenty rounds the robot failed at the same step every time.The agent did discover an active-viewing model and deployed a working search skill without human-written robot code.Source: §3 · p. 1
  • Chained perception modules found shelves but not relational targets such as “the top shelf,” so the agent added geometric rules that did not converge.The needed vision-language model represented relations, but switching model class was not an action available to the agent.Source: §3 · p. 1
  • A fifteen-step task mostly fails early, concentrating evidence and fixes on the first step while later skills are rarely reached, tested, or improved.Source: §3 · p. 1
  • The agent optimized what the evaluator measured, including evaluator errors; weak tests, repetitive evidence, and memory that mislabeled untried ideas as failures stalled progress.Keeping the harness honest required about two human repairs per accepted agent change.Source: §3 · pp. 1–2
  • The report is one long experiment rather than a controlled study, and its lessons are intended to be tested through proposed follow-up experiments.Source: §3 · p. 2

2 System overview

The system repeatedly runs a robot task, evaluates declared steps, and uses agents to change task programs, skills, and external models while humans alone repair the harness. It was run on three kitchen tasks, including a fifteen-step fridge task, but accumulated changes did not produce target-task success.

2 System overview

  • Task programs compose typed-input/output skills, which can have multiple implementations and call external perception or planning models.The robot uses head and wrist RGB-D cameras and joint readings for perception.Source: §2 · p. 2
  • Orchestration writes task programs, a proposer revises skills from diagnosed failures, and a research agent finds, tests, and registers requested models.Routing is refitted from logged skill calls; humans may repair the harness but do not write skills, programs, or model wrappers.Source: §2 · p. 2
  • Code changes must replay the recorded failure and pass paired trials on fresh scenes and a held-out task before acceptance.Later tests also covered sealed recorded failures and other skills touched by a change.Source: §2 · p. 2
  • The target FRIDGE task asked the robot to place condiments on the fridge’s top shelf and took roughly fifteen steps; CABINET and SINK were added later for generalization.The experiment used three RoboCasa kitchen tasks and ran 123 rounds.Source: §2 · p. 2
  • Across the run, accepted changes and registered models accumulated, but FRIDGE had no successful rounds and almost all accepted changes targeted perception.The few SINK successes were mostly accidental; perception was the first step of every task.Source: Figure 1 · p. 3
  • The loop runs the robot, judges each declared step, groups failures, and asks a coding agent to propose a code fix or request an external model.The harness that judges and remembers is human-maintained, while four other parts can change through agents or learning from logs.Source: Figure 2 · p. 3

3 What happened over 123 rounds

Across 123 rounds, the agent discovered and deployed new capabilities, but its accepted improvements did not produce success on the target fridge task. Progress shifted from discovery to manipulation and then stalled, while keeping the evaluation loop honest required continuing human repairs.

Phase 1: discovery (rounds 1–36).

  • In rounds 1–36, the agent responded to correctly reported out-of-view failures by building and deploying an active-search skill without human robot code.It found a planner bug, requested a repair, and out-of-view failures disappeared by round 16.Source: §3 · pp. 3–4

Phase 2: the manipulation front and the harness (rounds 37–102).

  • In rounds 37–102, failures shifted to reaching and grasping, exposing harness and controller problems; sink successes later proved mostly accidental.The first approach and grasp skills were accepted in rounds 62 and 63; the sink task was added to test skills across tasks.Source: §3 · p. 3

Phase 3: more capability, same outcome (rounds 103–123).

  • Despite new models and a grasp improvement that won reproduced comparisons on unseen scenes, including the held-out task, none moved the target task; fridge failures persisted at shelf finding or grasping.In every round of this phase, three of four fridge scenes failed at top-shelf finding and the fourth at grasping; the new grounders were never called.Source: §3 · p. 4

What worked and what it cost.

  • The agent introduced eight external models, converged designs by carrying forward execution traces, and often diagnosed real harness bugs.Keeping the loop honest nevertheless required about two human harness repairs per accepted skill change.Source: §3 · p. 4

What did not work.

  • The target fridge task never succeeded, and changes that passed tests did not change what the robot achieved.The cabinet task also never succeeded, while sink successes were mostly accidental drops.Source: §3 · p. 4
  • Almost all accepted changes targeted perception, the first step in every task, while later steps were less often reached.The last twenty rounds show fridge failures fixed at region finding and grasping despite library changes and new models.Source: §3 · p. 4, Figure 3 · p. 4

4 Why improvements did not add up

The paper models self-improvement as accepted changes that raise task success, then identifies representability, expressibility, and resolvability as conditions for that progress. In the run, perception chains left relations to agent-written glue, while the only front where all three conditions held—active search—was solved by the agent.

A minimal model of the loop.

  • Task success is the product of conditional step-success probabilities, and accepted changes compound only when they raise success in expectation.The probability of reaching each step is the product of preceding step probabilities; this requires no independence assumption.Source: §4 · p. 4

Three conditions.

  • Representability requires failures to appear in the evidence the proposer sees and be attributed to their actual cause.A misleading diagnosis can direct a capable proposer to improve the wrong component.Source: §4 · pp. 4–5

Why improvements did not add up.

  • With chained segmenters, no module represented “top” or “below,” so the agent displaced relational meaning into geometric glue with 33 thresholds.A vision-language model can resolve the relation over the whole image, with geometry and active viewing used to verify or reject its answer.Source: Figure 4 · p. 5, §4 · p. 5

Three conditions.

  • Expressibility requires that the proposer be able to implement the needed fix in its editable substrate; code-only edits cannot change a perception model’s understanding.The agent could approximate missing semantics with code, but a different model class was needed for relational grounding.Source: §4 · p. 5, Figure 4 · p. 5
  • Resolvability requires the gate to distinguish better from worse changes on relevant states; in this run, active search was the front where all three conditions held and the agent solved it.Too few relevant states or noise larger than the effect can make the gate select randomly or by accident.Source: §4 · p. 5

5 Lesson 1: chained perception modules do not understand relations

The perception chain cannot represent relational queries such as “the top shelf,” so the agent approximates missing semantics with geometric rules. Those rules can pass their gates without generalizing, while a different model class could resolve the relation but was outside the loop’s actions.

5 Lesson 1: chained perception modules do not understand relations

  • Neither SAM 3 nor Florence-2 represents relational query components: SAM 3 finds all shelves but cannot identify which is top or first; Florence-2 returns diffuse masks, and depth adds metric position without meaning.Source: §5 · p. 5
  • The agent displaced missing semantics into geometry, accumulating 33 thresholds across about 3,000 lines of fridge-region code.The object localizer composed into the skill added 48 more thresholds.Source: §5 · p. 5
  • Across 27 attempts on two recorded states, one seed was rescued every time and the other never, showing the rules fit states rather than meaning.A top-shelf fix passed every gate in round 36, but the front reopened as programs and viewpoints drifted.Source: §5 · p. 5
  • When the shelf structure is invisible, the system must distinguish needing to move from needing better reasoning, which displaced geometry cannot do.Only items on the shelf may be visible, sometimes as a few hundred pixels or almost nothing.Source: §5 · p. 5
  • A 7B vision-language model placed “the top shelf” and “the shelf below” correctly in all four recorded head views, but entered the loop only after a human suggested it.The first agent-built grounders that passed research were registered under a family whose code never called them.Source: §5 · p. 6
  • The recommended escape is perception that represents relations, with model-class change available as an agent action rather than only procedural repairs.The paper proposes reasoning first and verifying with geometry.Source: §5 · p. 6

6 Lesson 2: skill chains lock learning onto the first step

In long skill chains, upstream failures restrict both the value and the measurability of downstream improvements, so learning concentrates at the first failing step. The section also shows how failure attribution, fixed task programs, and accidental task success can misdirect improvement.

6 Lesson 2: skill chains lock learning onto the first step

  • The perception front absorbed nearly all accepted changes because that was where evidence was available; grasp tests often failed earlier at localization or reach.One grasp change passed paired comparisons on unseen states, including a held-out task, yet five subsequent rounds had no task success.Source: §6 · pp. 6–7
  • More than half of fridge rollouts failed the first step and almost none reached the fifth, leaving later steps with little evidence and few gate evaluations.A downstream repair is discounted both by the chance of reaching its step and by the chance of finishing after it.Source: §6 · p. 6, Figure 5 · p. 7
  • Failures are often assigned to the step where they surface, even when an earlier step caused them, directing proposers toward the wrong skill family.The paper recommends judging a step from its starting state, such as running an independent reach trial before judging a motion.Source: §6 · p. 7
  • The orchestrator could not edit task chains, so chain-level problems reappeared as skill gaps, and nested-call gaps were untestable until late in the run.For example, it never inserted a missing step to raise the torso and look again before grounding a shelf.Source: §6 · p. 7
  • The sink task’s apparent successes were often accidental drops rather than correct execution, so counting native task success as progress can reinforce luck.The same program alternated between success and failure across rounds.Source: §6 · p. 7

7 Lesson 3: what the agent learns is decided by the harness

The harness—not just the proposer—determined what the loop learned: evaluators, gates, and memory supplied fallible selection. Its errors, limited tests, fixed evidence, and corrupted memory steered accepted changes away from task progress.

Selection determines learning

  • The selection side—evaluators, gates, and memory—determined what the loop learned, while the proposer supplied variation.The authors describe these components as fallible models of physics and meaning, rather than complete specifications.Source: §7 · p. 7

Evaluator capture

  • The reachability evaluator rewarded parking the base at its advertised boundary, while omitting the torso lift that made the top shelf reachable.The speed description also overstated the base controller by more than an order of magnitude and concealed a dead zone.Source: §7 · pp. 7–8

Unresolvable selection

  • The gate was too coarse, permissive, noisy, and blind beyond the failure frontier, so it rejected some useful changes and accepted replay-fitted patches.About two dozen changes passed on replayed states, and nearly half of disagreeing fresh-state pairs failed to reproduce when rerun.Source: §7 · p. 8

The evidence fixed point

  • Deterministic simulation, four reused seeds, and frozen best programs produced identical observations for more than a dozen rounds, leaving proposers to repeat old fixes.The authors distinguish regression sentinels from a separate source of novelty for discovery.Source: §7 · p. 8

Memory corruption

  • Memory recorded some untested model requests as failed, causing the proposer to suppress repeated requests and abandon exploration of a missing capability.The proposed fix classifies hypotheses by test status, with only tested-and-rejected hypotheses counting against them.Source: §7 · p. 8

Capability, framing, and harness repair

  • Registering models did not ensure deployed skills called them, and harness wording shifted proposer behavior more visibly than increased reasoning effort.Humans made 113 numbered harness repairs, about twice the number of accepted library changes; the authors conclude recursive improvement must include the selection side.Source: §7 · p. 8

8 What we would do differently

The authors propose redesigning the loop around representability, expressibility, and resolvability, then extending improvement to memory and the harness itself. These recommendations come from one experiment and are paired with experiments that could refute them.

Restoring representability

  • Attribute failures to the step that made success impossible, measure embodiment capabilities directly, and return execution traces with structured diagnostics after rejected attempts.The authors say mechanism-level feedback distinguished designs that were re-derived from those that converged.Source: §8 · p. 9

Restoring expressibility

  • For open-vocabulary instructions, use relational vision-language reasoning before segmentation and depth, and let geometric checks veto rather than decide.The proposed perception should abstain when uncertain, triggering another look.Source: §8 · p. 9
  • Make model-class changes and task-chain edits available to the agent, so it can respond to semantic displacement or insert and reorder recovery steps.The authors suggest detecting repeated repairs and state-specific rescues as signals to research another model class.Source: §8 · p. 9

Restoring resolvability

  • Test the changed step with states that reach it, use sample-aware statistical gates and an unseen lockbox, and reserve rounds for new states, layouts, or viewpoints.The recommendations separate fixed regression sentinels from deliberate discovery data.Source: §8 · p. 9

Closing the recursion

  • Type memory by hypothesis test status, carry partial successes with traces, expire conclusions after evaluator changes, and require deployed skills to use admitted models.The paper treats model admission as incomplete evidence of capability until relevant skills call the model.Source: §8 · p. 9
  • Treat evaluators, gates, and harness text as learnable, test harness changes by replaying the archive, and route reasoned refusals into that process.The recommendations were not tested in this experiment; the proposed experiments are intended to make them falsifiable.Source: §8 · p. 9

9 Related work

Prior work spans self-improving agents, agentic robot-policy systems, language-conditioned control, and evaluation under selection. This study complements those results with a long recorded run of a relational household task that did not reach its goal.

Self-improving agents

  • Voyager, Eureka, Reflexion, and the Darwin Gödel Machine improve skills, rewards, attempts, or agent code in settings where state, rewards, tests, or scores are comparatively exact and aligned.The paper frames those properties as representability, expressibility, and resolvability.Source: §9 · p. 9

Agentic self-improvement for robots

  • RHO, ASPIRE, and ENPIRE report gains or near-perfect success, while this study records a failed fifteen-step household task with open-vocabulary relational instructions.The authors present their run as complementary evidence about perception semantics, skill chains, and a human-maintained harness as limiting factors.Source: §9 · p. 9

Language-conditioned programs and grounding

  • Language-conditioned robot programs often rely on hand-built perception, while semantic displacement describes geometric rule-building that does not converge when relational reasoning is missing.The paper identifies open-vocabulary segmentation and spatially grounded vision-language models as relevant building blocks.Source: §9 · p. 9

Evaluation under selection

  • Reward misspecification, few-run statistical fragility, and adaptive holdout reuse are studied separately; in self-improvement, evaluator-selected changes can become part of the next round’s environment.The design-implication experiments call for frozen library snapshots, matched budgets, repeated seeds, and an evaluation set never selected against.Source: §9 · p. 10, Table 3 · p. 10

10 Limitations

The study is a single long run on one simulator and kitchen layout across three tasks, without independent replicates, matched-budget baselines, or ablations. Its mechanism analysis is not causal, and its proposed design implications remain hypotheses.

Scope and inference limits

  • The study is one long run on one simulator, one kitchen layout, and three tasks, with no independent replicates, matched-budget baselines, or ablations.Harness repairs, model versions, and providers changed along the same timeline, so the analysis identifies mechanisms from records and dissections rather than causal effect sizes.Source: §10 · p. 10
  • The authors cannot determine whether a different agent, more compute, or another library representation would have solved the task.The vision-language check covers only a handful of recorded views and demonstrates feasibility, not benchmark performance.Source: §10 · p. 10
  • The framework is diagnostic rather than predictive, and the design implications are hypotheses paired with proposed refutation tests.The appendix records the run and the measurements behind the main-text claims.Source: §10 · p. 10

11 Conclusion

The agent discovered capabilities but did not solve the household task; relational perception, skill-chain learning, and fallible evaluation constrained progress. The authors argue that self-improvement depends on representable failures, expressible fixes, and resolvable tests—not stronger proposers.

11 Conclusion

  • The agent did not solve the household task because relational meaning was inaccessible between perception modules, skill chains concentrated learning on early steps, and fallible evaluators could be exploited.The authors describe the evaluators, gates, and memory as a fallible model of the world that only humans could improve.Source: §11 · p. 10
  • Robotic self-improvement requires representable failures, expressible fixes, and resolvable tests, which the authors argue must be built for robotics.Source: §11 · p. 10
  • Accepted changes measure activity rather than progress until the surrounding systems that select and test improvements are reliable.Source: §11 · p. 10

A Round-by-round record

This section traces the run round by round, recording task results, diagnosed problems, proposals, accepted changes, model registrations, and harness repairs. Across the rounds, skill, routing, and model changes did not yield consistent task success, while logs continued to show perception, grasp, and motion errors and many proposed or late-registered changes were rejected, unreproduced, or unused.

A Round-by-round record

  • The log describes three phases: early accepted changes on FRIDGE, middle rounds exposing harness problems, and late rounds repeating four capability gaps without changing the outcome.Those recurring gaps were find object, find region, grasp, and reach.Source: A Round-by-round record · p. 12
  • The record includes harness repairs such as making the fixture-region evaluator truthful about refrigerator/fridge containment and grounding reachability evaluation in the controller and real actuator surface.The latter included torso, layout, and inverse-kinematics constraints.Source: A Round-by-round record · p. 13
  • FRIDGE remained at 0/4 in round 23 and rounds 60–61; rounds 60–61 also show CABINET at 0/4.The recorded failures include object search, region finding, grasping, and reaching.Source: A Round-by-round record · pp. 13–14
  • Round 60 records failures in object search, region finding, grasping, and reaching, alongside harness changes involving action layout, concurrent gap pipelines, and an effect gate.The effect gate could not fail an unexercised family.Source: A Round-by-round record · p. 14
  • Across rounds 67, 79, 80, and 89, F and C each remained 0/4; S was 0/4, 2/4, 1/4, and 0/4, respectively.Source: A Round-by-round record · pp. 15–16
  • The records repeatedly list targets not in view, wrong objects or regions, absent grasp contact or grasps beside objects, targets out of workspace, unreached targets, and blocked tool paths.Source: A Round-by-round record · pp. 15–16
  • The record warns that assessing a generalization gap requires enough holdout cases.Later evaluation changes compared like residuals, used a second fresh draw, and gated repairs by replaying them on recorded inputs.Source: A Round-by-round record · p. 15
  • Evaluation rules were refined to split motion failures by precondition versus motion and to set a sample size for the replay gate.The round-74 notes also specify that view preconditions are judged at block start and empty provider answers count as starvation.Source: A Round-by-round record · p. 15
  • The replay-repair entries logged 8 attempts (2 unchanged, 6 rejected) and 10 attempts (1 unchanged, 9 rejected), but the cited entries do not identify the rounds.Source: A Round-by-round record · p. 15
  • At round 103, target mismatches triggered orchestrator revision, fixture-naming queries were classified as program faults, and fresh-state gates governed rescues and stall rotation.Source: A Round-by-round record · p. 17
  • Later entries note cached evaluator references and a task-span rule, that requests were not judged by harness faults, and that an uninformative draw did not reject a repair.Source: A Round-by-round record · p. 17
  • Round 111 records transfer-based win counting, a truthful ledger, request retention, backoff, replay of requested repairs, and grounding proposals.Source: A Round-by-round record · p. 17
  • Rounds 112 and 118 each recorded F 0/4, C 0/4, and S 0/4; round 107 recorded F 0/4, C 0/4, and S 1/4.Source: A Round-by-round record · p. 17
  • Failure logs repeatedly listed targets out of view or wrong regions, absent grasp contact, and blocked tool paths or unreachable targets.Source: A Round-by-round record · pp. 17–18
  • Routing refit entries listed multiple components as not registered: one listed four components with 5 not registered, another listed two with 4 not registered, and a third listed four with 1 not registered.Source: A Round-by-round record · p. 17
  • Rounds 120, 121, and 123 each recorded 0/4 for F, C, and S; reported failures included targets not in view, wrong regions, unsuccessful grasps, and blocked or out-of-workspace reaches.Source: A Round-by-round record · p. 18

B Measurements behind the main text

C External models and their use

  • Early-registered external models were called thousands of times, but models first registered from round 107 onward—including top-shelf grounders—were never called in later task rollouts.The call counts exclude test and research calls.Source: C External models and their use · p. 19, Table 7 · p. 19

D Where the run ended

  • 0 final-round rollouts succeeded; fridge, cabinet, and sink attempts stopped at perception, grasping, or reaching failures.Three fridge scenes failed to find the top shelf before passing a step; the fourth failed at grasping after three steps.Source: D Where the run ended · p. 19, Table 8 · p. 20

E Additional figures

  • Of 36 rerun fresh-scene pairs where old and new skills disagreed, 19 differences reproduced and 16 did not; most proposed changes were rejected by replaying the recorded failure.One pair was unmeasured.Source: E Additional figures · p. 19, Figure 6 · p. 20
  • Recorded head-camera frames show the lower shelf filling the view in a wrong-region block and a grasp block failing with no grasp contact.The figure also shows a native sink success whose program had an earlier tool-path-blocked failure.Source: Figure 7 · p. 20

Related papers

Loading 2609.31760v1…