RoboDoctor: Training-Free Recursive Self-Improvement for Robot Policies

1 INTRODUCTION

• Robot capability depends on coordinating, evaluating, and revising available behaviors because each physical interaction changes the conditions for later actions.The motivating question is how execution feedback can adapt the deployment of existing model capabilities.Source: §1 · p. 2
• Revision must target the level implicated by feedback: a local alignment correction can preserve a useful grasp, while repeatedly restoring that grasp can perpetuate an ineffective strategy.The challenge is to identify what needs revision without discarding behavior that remains useful.Source: §1 · p. 2
• With policy and agent weights fixed, physical interaction both advances the task and provides evidence for revising subsequent decisions and accumulated methods.The adaptive state lies in control organization, intervention decisions, and experience that informs later behavior.Source: §1 · p. 2
• Reuse links a decision's rationale, physical method, observed effect, and applicability conditions; methods must be grounded again in the current scene.A successful correction supports reuse only under conditions justified by observation, and its original displacement may not transfer.Source: §1 · p. 2
• RoboDoctor couples three revision targets: control allocation between policy and agent, selective intervention that preserves useful policy behavior, and refinement of physical operations.Its linked experience records connect a decision's reason to the attempted method, observed outcome, and conditions for reuse.Source: §1 · pp. 2–3
• 57.10 Overall score in Refinement, compared with 49.74 during Warmup, across a curated archive of 54 RoboDojo-Sim task entries.The largest gains occur in Generalization and Open instruction following, while Long-Horizon performance decreases.Source: §1 · p. 2

2 RELATED WORK

• Generalist policies and world action models provide manipulation behaviors, while language-model agents contribute task interpretation, planning, and tool use.The cited work includes policies for visuomotor control and generalist manipulation, alongside models that connect action generation to physical evolution.Source: §2 · p. 3
• Robot-agent work incorporates feedback into planning, failure explanation, recovery, and test-time action selection; RoboDoctor studies assistance to an existing continuous policy.Its focus is which policy behavior to preserve, when to replace a proposal, and whether a correction addresses the task limitation.Source: §2 · p. 3
• Embodied RSI systems revise harnesses, co-evolve code and skills, use shared stateful code libraries, or target corrective data collection and policy training.RoboDoctor instead places adaptation in control allocation and external experience with both model weights fixed; its harness selects between predefined controllers.Source: §2 · p. 3
• Fixed-weight agents use external experience in forms such as verbal feedback, executable skills, interaction memory, and context organized from demonstrations and execution feedback.RoboDoctor focuses on the physical evidence that makes a lesson applicable: its diagnostic trigger, retained policy behavior, attempted operation, and observed effect.Source: §2 · p. 3
• After three HYBRID warmup episodes, Harness RSI selects HYBRID or DIRECT control and keeps that choice fixed.HYBRID retains up to 16 unchanged policy steps or issues a grounded correction of 1–5 physical steps; both correction and DIRECT actions verify observed effects.Source: Figure 2 · p. 4

3.1 EXECUTION-TIME ADAPTATION WITH FROZEN MODELS

• RoboDoctor uses fixed models and external experience to connect current observations with control allocation, intervention, and grounded operation.The Doctor receives RGB views, metric depth, joint state, and end-effector poses without privileged object poses or object-state labels; the policy proposes action chunks from the instruction and observation history.Source: §3.1 · p. 4

3.2 HARNESS RSI: ALLOCATE CONTROL RESPONSIBILITY

• After three HYBRID Warmup episodes, coarse native outcomes determine whether the run continues HYBRID or switches to DIRECT, and the selected controller remains fixed.HYBRID preserves its intervention experience; switching to DIRECT initializes a separate task-experience document. The gate allocates control coarsely rather than diagnosing a specific failure.Source: §3.2 · p. 5

3.3 INTERVENTION RSI: ADAPT ASSISTANCE TO EXECUTION EVIDENCE

• In HYBRID, the Doctor reviews a policy proposal and either releases an unchanged prefix or substitutes a grounded correction, then obtains a fresh observation.Executed frames, including corrections, enter policy history; later observations can reveal missed failures or disturbed grasps and inform subsequent assistance while autonomous successes identify behavior to preserve.Source: §3.3 · p. 5

3.4 OPERATION RSI: GROUND, TEST, AND REVISE PHYSICAL METHODS

• Operation RSI grounds a physical method in visible surfaces, depth, measured poses, and current views, then uses execution evidence to revise the method and verify its effect.A localization reference is not necessarily a valid contact or end-effector target; failure to reach a target points toward motion or configuration, while reaching it as the object slips suggests revisiting contact technique. HYBRID retains a five-step per-call limit, and DIRECT can vary budgets by motion type.Source: §3.4 · pp. 5–6

3.5 EXPERIENCE: TURN PHYSICAL FEEDBACK INTO REUSABLE METHODS

• Persistent experience links a decision’s reason, attempted method, observed effect, and applicability, then re-grounds retained lessons against fresh scene geometry before reuse.HYBRID records can be appended or revised subject to structural validation, which checks required fields rather than independently proving diagnoses or resolving contradictions. DIRECT consolidates lessons after termination; both modes adapt through external state under fixed weights, without guaranteeing improvement.Source: §3.5 · p. 6

4.1 BENCHMARK AND EVALUATION PROTOCOL

• The archive covers five dimensions, and Warmup already includes Doctor assistance and experience updates rather than serving as an unassisted or no-memory baseline.Selected valid outcomes may come from different source runs and memory histories; reference methods also have unmatched budgets and selection procedures.Source: §4.1 · p. 7

4.2 AGGREGATE PERFORMANCE AND CAPABILITY PROFILE

• Generalization and Open improve by 9.48 and 21.35 points, while Precision and Memory improve by 4.06 and 3.61 points; Long-Horizon falls from 54.58 to 52.92.Memory performance does not independently measure the experience mechanism’s contribution.Source: §4.2 · pp. 7–8, Table 2 · p. 8
• 57.10 overall score and 49.44% native success in Refinement exceed HexaShell-Preview’s 53.18 and 46.85%, respectively, under unmatched protocols.RoboDoctor’s selected local outcomes rise from 49.74 overall score and 40.83% success in Warmup; reference interaction budgets and evaluation protocols differ.Source: §4.2 · p. 7, Table 1 · p. 8

4.3 TASK-LEVEL VARIATION AND INTERPRETATION

• Across 54 entries, 24 task means improve from Warmup to Refinement, 20 decline, and 10 remain unchanged.Figure 3(b) orders entries by change; examples include Align Blocks rising from 0 to 100 and Play Xylophone falling from 100 to 50.Source: §4.3 · pp. 8–9, Figure 3 · p. 9
• The stage comparison combines additional interaction, experience revision, route selection, and source-run differences, so it cannot isolate any component’s contribution.The paper identifies matched experiments controlling routes, seeds, and interaction budgets as necessary to establish causal contributions.Source: §4.1 · p. 7, §4.3 · p. 9

5.1 HARNESS: REVISING THE INTERACTION STRATEGY

• Supported table contact and coordinated two-gripper pushing replaced an unresolved attempt, illustrating Harness RSI's shift to an agent-led interaction strategy.The later strategy used spatially separated contacts along the tool edge.Source: §5.1 · p. 9, Figure 4 · p. 10

5.2 INTERVENTION: CORRECTING ALIGNMENT WHILE PRESERVING THE GRASP

• A one-step lateral correction aligned the bottle while preserving its grasp, after which the policy resumed tilting, poured, and withdrew the bottle.The case illustrates selective assistance at the alignment level while retaining useful policy behavior.Source: §5.2 · pp. 9–10, Figure 4 · p. 10

5.3 OPERATION: REVISING AND VERIFYING THE GRASP

• After slippage, the Doctor revised the grasp and used a test lift to check retention before WAM-led placement.The check provided evidence about grasp retention; placement separately showed progress toward the task objective.Source: §5.3 · p. 10, Figure 4 · p. 10

• Perceptual errors, incorrect diagnoses, and overgeneralized lessons can persist, and schema validation makes records inspectable without establishing their truth.A plausible command or achieved pose does not certify stable contact, and visible progress may not satisfy the native task predicate.Source: §6 · p. 10
• The selected cases and aggregate results characterize runs but do not establish that cross-episode experience caused improvement; retrospective selection and unequal coverage constrain attribution.The proposed prospective comparison would match controllers, configurations, seeds, and interaction budgets.Source: §6 · pp. 10–11
• The archive lacks a consolidated cost analysis, all results are in simulation, and transfer to other policies or physical robots is unmeasured.Bounded actions limit motion before renewed observation but provide neither collision avoidance nor a formal safety guarantee.Source: §6 · p. 11

• RoboDoctor connects control allocation, selective intervention, and operation revision through experience retaining methods and supporting evidence while model weights remain fixed.The archive shows a higher aggregate Refinement score alongside concrete execution changes and regressions; matched experiments are needed to test when revised experience improves subsequent execution.Source: §7 · p. 11
• Observed effects inform what to preserve, revise, and reuse in later attempts, while the evaluated archive characterizes execution-time adaptation rather than isolating the causal contribution of cross-episode experience.The next experimental priority is testing revised experience under matched configurations and interaction budgets.Source: §7 · p. 11
• The archive shows a higher aggregate Refinement score alongside concrete strategy changes and local corrections, with regressions that expose remaining limitations.Source: §7 · p. 11

• Policy rollouts validate a finite H ×14 proposal and preview it without physical steps; continuation requires its proposal identifier and executes an unmodified, capped prefix.A new observation clears the old proposal, and HYBRID direct-tool interventions require an existing proposal and a one-to-five-step budget.Source: A RUNTIME CONTRACTS AND EXTERNAL… · p. 14
• The host caps released HYBRID prefixes at 16 steps, below the bundled preview backend's 24-step horizon and 32-action prediction length.The preview uses four past RGB frames at stride four plus the current frame; these settings do not establish identical configurations across archived runs.Source: A RUNTIME CONTRACTS AND EXTERNAL… · p. 14
• Grounded-action checks distinguish levels of support, but none replaces the final native task outcome.Source: Table 3 · p. 14
• HYBRID experience retains diagnosis evidence and operation details, including observed outcomes and applicability conditions, but its natural-language interpretation is not mechanically verified as true.Unknown geometry remains qualitative or unknown.Source: A RUNTIME CONTRACTS AND EXTERNAL… · p. 14
• DIRECT carries prior lessons forward as reference material, while each new episode must reconstruct current progress, object poses, and absolute coordinates from fresh observations.Post-episode consolidation is action-disabled, and the host appends the native outcome.Source: A RUNTIME CONTRACTS AND EXTERNAL… · p. 14
• DIRECT permits up to 150 steps for coarse motion on a confidently clear route, whereas HYBRID interventions retain a five-step cap.Guidance about contact regions, test lifts, arm roles, and viewpoint changes is supplied before interaction rather than newly discovered in every run.Source: A RUNTIME CONTRACTS AND EXTERNAL… · pp. 14–15

• Table 4 reports Warmup score (W), Refinement score (R), and their difference, ∆ = R − W, for all 54 task entries.Source: Table 4 · p. 15
• For each task, SR is the Refinement success rate, while n counts valid Refinement episodes out of six nominal slots.Source: Table 4 · p. 15
• All tasks have three Warmup outcomes; Route identifies selected Refinement records, and Mixed marks retrospective combinations of both modes.Source: Table 4 · p. 15

• The examples connect revised interactions to depicted outcomes, but represent distinct interactions rather than one continuous episode or measured cross-task experience transfer.Source: C CASE EVIDENCE AND INTERPRETATION · p. 16, Table 5 · p. 16
• Figure 2 uses pouring to show how a local correction supports subsequent policy motion, while experience fields and update rules are specified elsewhere.Source: C CASE EVIDENCE AND INTERPRETATION · p. 16
• Liquid entering a cup and a block remaining held during a test lift evidence depicted interactions, not evaluator-defined task success or experience-reuse benefits across episodes.Source: C CASE EVIDENCE AND INTERPRETATION · p. 16