PaperlayerView exact revision on arXiv

Source-linked AI summary

Where is Physical RSI capability from? An Observational Scaling Law of LLM-as-policy

Zekai Li, Jiayi Yu, Jinlu Zhang, Luyang Niu, Zekun Qi, Jun Zheng, Xuchuan Chen, Dairu Liu, Yintianrun Zhang, He Wang, Li Yi

arXiv:2610.26848v1cs.ROcs.AIcs.CV

TL;DR

The paper asks where physical recursive self-improvement comes from, how to measure and predict it, and whether gains generalize. It introduces PhysRSI-eval and studies capability associations, scaling, and workspace interventions; its findings indicate that high scores, early gains, and efficient interaction need not coincide, while retained experience can remain useful beyond practice scenes.

  • Problem

    Physical RSI raises questions about capability sources, how scaling relates to improvement, and whether gains reflect overfitting or generalization.

    Source: §1 · p. 1

  • Method

    PhysRSI-eval measures physical RSI across five axes and studies capability associations, observational scaling, early-feedback forecasts, and workspace interventions.

    Source: §1 · p. 1, §3.4 · p. 5, §5.2 · p. 7

  • Results

    The five-axis results show that higher attained scores need not coincide with earlier gains or fewer physical interactions.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Workspace interventions support a role for retained experience and show that it can remain useful on new layouts and tasks, so scene-specific adaptation and reusable improvement can coexist.

    Source: Q3: Does Physical RSI reflect… · p. 2, §8 · p. 12

    Capability analyses use public benchmarks with differing evaluation settings, cover six systems without independent validation, and remain hypotheses to test.

    Source: §7 · p. 12

Abstract

from arXiv · show

In this paper, we systematically investigate the mechanisms, scaling behavior, and generalization of physical Recursive self-improvement. We introduce PhysRSI-eval to quantify physical RSI through a five-axis metric: First-Attempt Performance, Efficiency, Success, Budget and Improvement from failure. Different VLMs are evaluated on 18 manipulation tasks under a fixed interaction budget. The analysis of experimental results motivates the Physical Barrel Hypothesis: different capabilities may constrain different parts of improvement. The five-axis metrics also distinguish systems with close public benchmark averages and show that high scores need not come with early or efficient gains. We study observational scaling from two perspectives: predicting the relative performance of VLM agents from their public capabilities, and forecasting later progress from early physical feedback. Together, these analyses ask how quickly and how far an agent can improve. Moreover, we conduct workspace interventions and then analyze what improvement leaves behind. Removing history reduces late-round success, frozen workspaces remain useful on unseen layouts, and three cross-task cases achieve success earlier. The results suggest that scene-specific adaptation and reusable experience can coexist.

1 Introduction

The paper asks where physical recursive self-improvement (RSI) comes from, how to measure and predict it, and whether gains generalize beyond practice scenes. PhysRSI-eval studies capability associations, improvement under interaction budgets, and workspace interventions.

1 Introduction

  • PhysRSI-eval studies physical RSI through capability sources, five-axis quantification and observational scaling, and overfitting and generalization across tasks and scenes.Six systems revise persistent code, tools, and notes across 18 manipulation tasks under a fixed physical interaction budget; Gemini 4 is reported separately on ten tasks.Source: §1 · p. 1, Figure 1 · p. 2
  • An exploratory comparison associates spatial reasoning most strongly with initial performance, and perception with attained performance and recovery, motivating capability-specific bottlenecks rather than causal requirements.The authors call this the Physical Barrel Hypothesis: the capability that limits an agent may depend on which aspect of improvement is measured.Source: §1 · p. 1
  • The five-axis metric separates initial performance, gain timing, attained score, physical budget savings, and recovery from failure; higher scores need not mean earlier gains or fewer interactions.Opus 5.5 attains a higher score than GPT-6.1 Sol but has lower gain-timing and budget-savings scores.Source: §1 · p. 2
  • The observational scaling analyses ask whether public capability scores predict held-out systems’ relative performance and gain timing, and whether early physical progress forecasts later gains.The paper describes both analyses as exploratory.Source: §1 · p. 2

Q3: Does Physical RSI reflect overfitting or generalization?

  • Workspace interventions test whether retained experience helps beyond repeated attempts and whether it transfers to unseen layouts and other tasks.These tests address whether improvement reflects scene-specific adaptation or reusable experience.Source: Q3: Does Physical RSI reflect… · pp. 2–3
  • In a six-task Astra ablation, removing history lowers late-round success from 41.7% to 8.3%, while frozen workspaces support success on unseen layouts and three cross-task cases reach success earlier.The frozen-workspace and cross-task results suggest that scene-specific gains and useful transfer can coexist.Source: Q3: Does Physical RSI reflect… · p. 2

2 Related Work

Related work evaluates LLM agents in embodied settings, studies retained experience and self-improvement, and links public capability benchmarks to model performance. These strands motivate tracking physical progress and cost alongside testing whether workspace experience transfers.

2.1 LLM-as-policy Agents and Embodied Evaluation

  • Embodied evaluations typically summarize outcomes per episode, across independent trials, or within a time budget, while PhysRSI-eval connects judged physical attempts through a persistent workspace.A reported self-written fix that succeeded once but failed on repetition illustrates the distinction between finding a solution and retaining reliable behavior [9].Source: §2.1 · p. 3

2.2 Agent Self-Improvement

  • Prior self-improvement work examines harness changes, retained experience, memory, transfer, and gains relative to cost, motivating separate tests of improvement and what agents retain.Producing an update and benefiting from it are distinct questions [23].Source: §2.2 · p. 3

2.3 Capability Benchmarks and Observational Scaling

  • Public capability benchmarks cover perception, spatial reasoning, coding and tool use, while observational scaling relates benchmark-derived factors to performance across models.These analyses predict relative physical-improvement order and forecast later gains from early physical performance, but remain observational and exploratory.Source: §2.3 · p. 3

3 PhysRSI-eval: Benchmark Design

PhysRSI-eval separates first-attempt performance from improvement, its timing and physical cost, and tests what agents retain or transfer. It uses a controlled, fixed-budget loop across a difficulty-balanced set of manipulation tasks.

3.1 What Must the Benchmark Distinguish?

  • The benchmark tests whether retained code, tools, and notes help when cleared, remain useful on unseen layouts when frozen, or support new tasks when transferred.These workspace interventions are reported separately from the main profiles.Source: §3.1 · p. 3, §3 · p. 3

3.2 A Controlled Loop of Physical Attempts

  • At K0, each task is run three times without a persistent workspace; from K1, the agent revises a committed workspace after each graded physical attempt, stopping at success or H = 17 rounds.K0-to-K1 gains mix interface changes with improvement, so the Harness rounds are treated as the self-improvement segment.Source: §3.2 · p. 4

3.3 Tasks across Difficulty Levels

  • The 18 RoboDojo tasks cover five official categories and six difficulty strata, with three tasks per stratum ranked by official success rates before evaluation.Tasks use layout 0 and seed 0; all receive equal weight and retain native scores, success criteria, and step limits.Source: §3.3 · p. 4

3 PhysRSI-eval: Benchmark Design

  • PhysRSI-eval separately records starting performance, gain timing, attained score, physical budget savings, and recovery from failure.The profile distinguishes reaching a high score from reaching it early or using fewer physical interactions.Source: §3.1 · p. 3, §3.4 · p. 4

3.4 A Five-Axis Profile of Improvement

  • A1 is mean K0 score including partial progress, while A5 is the fraction of tasks failing at K1 that are later solved within budget.A5 is undefined when every task succeeds at K1; the five axes distinguish different aspects of improvement rather than collapsing them into an endpoint score.Source: §3.4 · pp. 4–5

4 Where is Physical RSI capability from?

The paper asks which capabilities relate to turning physical feedback into improved policies. In an exploratory comparison, spatial reasoning and perception track different axes, motivating a hypothesis of axis-specific capability bottlenecks.

4.1 Which Capabilities Track Physical RSI?

  • GPT-5.6 Sol scores higher than GPT-6 Astra on LiveBench code generation and completion, yet solves one of 18 tasks compared with Astra’s 15.The comparison cautions against treating coding benchmark strength alone as sufficient for physical improvement.Source: §4.1 · p. 5
  • Across six systems and 18 tasks, spatial reasoning has the strongest observed association with A1 (ρ = 0.94) and A2/A4 (ρ = 0.83), while perception leads for A3 and A5 (ρ = 0.94).Coding has the weakest association with every axis (ρ ≤0.37).Source: §4.1 · p. 6

4.2 The Physical Barrel Hypothesis

  • The Physical Barrel Hypothesis proposes that different improvement aspects may have different capability bottlenecks: spatial reasoning may matter more for starting well and perception for interpreting failures and recovery.These are candidate bottlenecks from associations, not causal requirements; weak coding associations do not mean coding is unnecessary, since all systems use code.Source: §4.2 · p. 6

5 How can we quantify Physical RSI and its scaling law?

The five-axis profile separates initial performance, gain timing, attained score, budget savings, and improvement, revealing trade-offs that a single score obscures. Public capability profiles predict both attained performance and gain timing across the six systems, while early feedback can forecast later progress.

5.1 High Scores, Fast Gains and Low Cost Can Diverge

  • Opus scores higher than GPT-6.1 Sol on Success (0.744 versus 0.703), but lower on Efficiency (0.609 versus 0.822) and Budget savings (0.579 versus 0.808).The gap persists on the ten tasks both systems solve; higher attained scores can come with later gains and greater physical costs.Source: §5.1 · p. 6, Table 1 · p. 7, Figure 5 · p. 8
  • Across six systems, task-mean scores rise from 0.01–0.37 at K0 to 0.10–0.84 by evaluation’s end, while public benchmark averages span 54.1–78.1 for the four GPT systems.Their equally weighted physical profile means span 13.9–73.4; GPT-5.6 Sol remains near zero cumulative success for 16 rounds.Source: §5.1 · pp. 6–7, Table 1 · p. 7
  • The Physical Barrel Hypothesis proposes that different capabilities may bottleneck different improvement axes, including a minimum needed capability despite strengths elsewhere.The threshold interpretation is motivated by the rapid contraction region; the authors frame it as a hypothesis, not a causal finding.Source: §5.1 · p. 7, Figure 4 · p. 7

5.2 Observational Scaling across Model Capabilities

  • The equal-weight average of four capability profiles reproduces the observed ordering of best K17 scores, and pairwise predictions recover every ordering of which system reaches T50 first.The T50 prediction uses a kernel regressor fitted on the other four systems’ profiles for each pair.Source: §5.2 · pp. 7–8, Figure 7 · p. 9

5.3 What Can Early Physical Feedback Predict?

  • The round reaching 90% of peak gain ranges from K7 for GPT-6.1 Sol to K17 for GPT-6 Sol and GPT-5.6 Sol.With K1–K5 observed, the Weibull template forecasts GPT-6.1 Sol through K17 with mean absolute error 0.019, versus 0.112 when holding the last observed score.Source: §5.3 · pp. 8–9

6 Does Physical RSI reflect overfitting or generalization?

Workspace interventions test whether improvement depends on retained experience and whether that experience remains useful beyond practice scenes. History supports late-round success, frozen workspaces work on unseen layouts, and cross-task cases show earlier success with transferred workspaces.

6.1 Does Retained Experience Help Beyond Repeated Attempts?

  • Removing all history lowers Astra’s late-round success from 15/36 (41.7%) to 3/36 (8.3%) across six tasks under the tested protocol.The full loop retains workspace, grader feedback, images, and action records; the no-history condition makes six independent attempts.Source: §6.1 · p. 9
  • Withholding grader feedback yields 5/36 (13.9%) late-round successes, while keeping only workspace and grader feedback yields 17/36 (47.2%).Past images and action records add little once those two channels are kept, according to the reported point estimates.Source: §6.1 · p. 9

6.2 What Survives When Revision Stops?

  • Frozen Astra workspaces succeed in 25/60 tests (41.7%) on unseen layouts without revision or outcome feedback; adding raw records yields 23/60 (38.3%).The comparison suggests scene-specific gains and transferable experience can coexist, but establishes cross-scene usefulness only for Astra on this task subset.Source: §6.2 · pp. 9–10, Figure 8 · p. 10
  • Pooled workspaces advance first success from K9 to K1 for Sonnet 5.5 on Cover Blocks, K14 to K7 for Opus 5.5 on Organize Table, and K15 to K8 for Opus 5.5 on Deposit Coin.These examples initialize each target task from a workspace pooled from all other tasks.Source: Figure 9 · p. 10

6.3 Can Experience Help on a Different Task?

  • Sonnet 5.5 and Opus 5.5 generate over 96% of later-round actions through self-written tools, whereas GPT-6 Sol’s tools generate 67%.Opus has broader tool-lineage reuse than Sonnet and GPT-6 Sol; Astra and GPT-6.1 Sol mostly generate actions directly while using memory, skills, and policy notes to guide revisions.Source: §6.3 · pp. 10–11, Figure 10 · p. 11
  • In Organize Table, Opus combines a parameterized move tool from Pack Objects Into Box with a grasp-orientation memory from Store Tools In Toolbox.The traces suggest semantic similarity helps identify candidate experience, while failure evidence guides whether to reuse, adapt, or discard it.Source: §6.3 · p. 11

7 Limitations

The evaluation is limited in system coverage and chain-level replication, and several analyses have narrower scope than the main comparison.

7 Limitations

  • The main comparison covers six model-plus-configuration systems with one improvement chain per task, so it does not estimate variability across complete chains.First-success stopping records one success; reliable repeated execution requires a separate test.Source: §7 · p. 12
  • Capability analyses rely on public benchmarks with settings that differ from the study’s runs and cover six systems without independent validation.The capability bottlenecks and gain-timing predictions therefore remain hypotheses to test.Source: §7 · p. 12
  • The ablation and frozen-workspace tests cover Astra and six tasks, while cross-task transfer results comprise three first-run cases.These limits constrain the scope of observed transfer and the mechanisms that can be inferred.Source: §7 · p. 12

8 Conclusion

The conclusion presents physical RSI as improvement that can combine scene-specific adaptation with reusable experience. It emphasizes measuring improvement across multiple outcomes and examining what agents retain.

8 Conclusion

  • PhysRSI-eval studies physical RSI’s origins, measurement, scaling, and generalization under a fixed interaction budget.Its five-axis metric separates initial performance, gain timing, attained score, physical budget savings, and recovery from failure.Source: §8 · p. 12
  • The Physical Barrel Hypothesis proposes that strengths needed to start well may differ from those needed to improve.The conclusion says high scores, early gains, and efficient interaction need not coincide.Source: §8 · p. 12
  • Cross-model prediction and early-round forecasting examine how improvement outcomes vary with capabilities and interaction budget.Source: §8 · p. 12
  • Workspace interventions support a role for retained experience, including usefulness on new layouts and tasks.The conclusion allows scene-specific adaptation and reusable improvement to coexist.Source: §8 · p. 12

A Task and Protocol Details

The protocol uses task-native grading and task-specific step limits, while separating fixed-layout evaluation from changing-layout ablation. Each action returns state and camera observations, with specified per-round and per-task budgets.

A Task and Protocol Details

  • The 18 tasks retain native grading and task-specific per-round step limits.The fixed-layout protocol and changing-layout ablation serve different comparisons and are reported separately.Source: A Task and Protocol Details · p. 16
  • Each action sets end-effector target poses within 5 cm translation and 0.35 rad rotation, runs 1–5 control steps at 25 Hz, and returns state JSON plus three camera images.Source: A Task and Protocol Details · p. 16
  • Per round, reasoning is budgeted at 4200 seconds and revision after grading at 1800 seconds for GPT systems or 600 seconds for Claude systems.The per-task physical budget is 17 full-step-limit attempts.Source: A Task and Protocol Details · p. 16

B Metric Definitions and Computation

The paper defines five-axis metrics from averaged task curves and reports curve summaries for the 18-task evaluation. Its analyses use public capability profiles and early physical scores to examine performance ordering and later trajectory forecasts.

B Metric Definitions and Computation

  • A1–A5 are computed after averaging task curves; Efficiency is the ratio of aggregate gains, not the average of task-wise ratios.The A3 threshold is the first integer Harness round reaching 90% of observed peak gain, and the summary is the equally weighted arithmetic mean of A1–A5 multiplied by 100.Source: B Metric Definitions and Computation · p. 16

C Detailed Experimental Results

  • The five-axis profiles use six systems across 18 tasks; Astra’s main-evaluation scores, curves, and costs exclude its separate no-hints experiment.The main evaluation solves 15 tasks, while the no-hints experiment solves 13; Table 5 reports each task’s first successful Harness round.Source: C Detailed Experimental Results · p. 16

D Capability and Prediction Analyses

  • Capability profiles average normalized ranks within four groups, then compare them with A1–A5 using Spearman correlation and adjusted permutation tests.The analysis covers 17 public benchmark scores and six systems; the smallest adjusted q is 0.11.Source: D Capability and Prediction Analyses · p. 16
  • Within-trajectory forecasts fit a template to Astra’s K1–K17 mean score, calibrate it to GPT-6.1 Sol’s early rounds, and forecast remaining rounds without updating on their outcomes.Figure 8’s band uses the 5th–95th percentiles of 300 paired task resamples.Source: D Capability and Prediction Analyses · pp. 16–17

E Ablation Details

The appendix details the ablation setup and information available across rounds, alongside the design and interpretation limits of frozen-workspace tests. It reports 22/36 late-round successes for Fixed on seen layouts, a descriptive result rather than a matched unseen-layout comparison.

E Ablation Details

  • Fixed has 22/36 late-round successes on seen layouts, but this is descriptive rather than a matched unseen-layout comparison.The six-task ablation runs all six rounds in each of two chains, without stopping after first success; Table 8 reports per-round successes.Source: E Ablation Details · p. 17, Table 8 · p. 18
  • Table 5 records each task’s first successful Harness round, with a dash indicating no success within K17.Source: Table 5 · p. 17

F Generalization Details

  • Frozen K6 workspaces are tested once on each of five held-out layouts without revision or outcome feedback, but outcomes sharing a source or task are dependent.The complete conditions include 12 source chains; the 60 source-layout outcomes are not 60 independent replications.Source: F Generalization Details · p. 17
  • Table 6 presents public benchmarks behind the capability profiles: scores were collected on 15 October 2026, and each group averages normalized benchmark ranks with equal weight.Source: Table 6 · p. 18
  • Earlier-round information varies by condition across workspace, grader, and image/action channels; current-round observations are available in every condition.The grader channel excludes step-level success flags in action replies and logs.Source: Table 7 · p. 18
  • The complete frozen-test conditions are comparable, whereas HO-NoImg and HO-NoGrader are incomplete and not comparable with them.Table 9 covers 12 sources × 5 held-out layouts for complete conditions.Source: Table 9 · p. 18

Related papers

Loading 2610.26848v1…