Source-linked AI summary
Agent Plasticity: Measuring Self-Improvement Through Experience
Harman Singh, Anton Bakhtin, Rulin Shao, Gabriel Synnaeve, Ilia Kulikov, Rob Fergus, Sanjeev Arora, Kurt Keutzer, Jason Weston, Anuj Mahajan, Anirudh Goyal
TL;DR
Fixed-point evaluations do not show whether agents improve and generalize through experience or how efficiently they acquire capabilities. The paper studies persistent self-improvement with inherited artifacts and measures performance across checkpoints. Across models and tasks, improvement, transfer, and plasticity vary, and artifact reuse alone does not guarantee success.
Problem
Endpoint performance alone cannot distinguish strong initial capability from improvement, account for differing experience costs, or show whether gains generalize beyond learning interactions.
Source: §1 · p. 2Method
The paper evaluates frozen-weight agents that inherit persistent self-constructed artifacts, measuring held-out performance across checkpoints and defining plasticity as experience-to-gain efficiency.
Source: §1 · pp. 2–3, §2.1 · p. 4Results
Across models and tasks, plasticity and persistent improvement vary substantially, gains need not transfer fully to harder settings, and artifact reuse does not guarantee success.
Source: §5 · p. 11Takeaways & Limitations
Evaluating long-lived agents requires measuring how capabilities change through experience and how efficiently those changes are acquired, not only fixed-point capability.
Source: §5 · p. 11The failure interpretations are observational rather than causal; controlled artifact interventions could help distinguish the proposed mechanisms.
Source: §4 · p. 10
Abstract
from arXiv · showhide
AI agents increasingly operate in environments where they can diagnose failures and improve through experience, yet existing evaluations largely measure what an agent can do at a fixed point in time rather than how effectively it learns. Evaluating self-improvement requires answering three questions: does future performance improve and generalize beyond the interactions that enabled learning; how efficiently are new capabilities acquired; and where does the self-improvement process break down? To answer these questions, we study self-improvement in a controlled setting where agents amortize past experience into reusable artifacts that are inherited by future instances. At each checkpoint, we measure performance on training and held-out environment interactions while accounting for learning cost. We introduce agent plasticity, the efficiency with which an agent converts experience into gains in future held-out performance. Across multiple environments, frontier models exhibit sharply different improvement trajectories despite comparable opportunities to learn. Some achieve substantial and persistent gains, while others remain near or below their initial performance, and gains within the training regime often transfer only partially to out-of-distribution conditions. Endpoint capability and acquisition efficiency also diverge: the agent that ultimately performs best need not be the one that improves most efficiently. Tracing failures through the improvement loop further reveals different candidate bottlenecks. Agents with low plasticity often fail to reuse relevant artifacts, whereas more plastic agents may still fail despite reusing relevant artifacts, pointing to limitations in artifact quality, generalization, or application. Evaluating self-improving agents requires measuring not only what they can do, but how effectively they become better through experience.
1 Introduction
Fixed-point evaluations miss whether agents improve, generalize, and acquire capabilities efficiently through experience. The paper measures these properties under a controlled protocol and finds substantial variation in improvement, transfer, plasticity, and artifact-use failures.
1 Introduction
- The evaluation asks whether future performance improves and generalizes beyond learning interactions, how efficiently capabilities are acquired, and where improvement breaks down.The paper frames these questions as gaps in endpoint performance alone.Source: §1 · p. 2
- The controlled protocol freezes model weights, gives future instances persistent self-constructed artifacts, and measures training and held-out performance at each checkpoint.Held-out ID tests generalization beyond learning interactions; held-out OOD tests transfer to harder regimes.Source: §1 · p. 2
- Agent plasticity measures how efficiently experience becomes held-out performance gains, separating acquisition efficiency from capability itself.The paper presents plasticity as one property of the improvement process, not as a definition of self-improvement.Source: §1 · p. 2
- Across environments, frontier agents show sharply different improvement trajectories, and held-out ID gains often transfer only partially to OOD conditions.Figure 3 shows checkpoint scores for Hard chess across training, held-out ID, and held-out OOD interactions.Source: §1 · pp. 2–3, Figure 3 · p. 3
- Endpoint capability and acquisition efficiency diverge: Claude Fable 5 reaches the highest fitted endpoint, while GPT-5.6 Sol achieves substantially greater estimated plasticity per learning cost.GPT-5.6 Sol also exceeds Claude Opus 4.8 in plasticity and eventual performance despite starting from lower initial capability.Source: §1 · p. 3
- Failure analysis suggests different bottlenecks: low-plasticity agents often fail to reuse relevant artifacts, while more plastic agents may fail despite reuse.Candidate bottlenecks include artifact retrieval and reuse, as well as artifact quality, generalization, or application.Source: §1 · p. 3
2 Measuring Self-Improvement through Experience
The protocol evaluates persistent self-improvement with fixed model weights, fresh contexts, and held-out performance measured across checkpoints and against learning cost. It measures acquisition efficiency and diagnoses whether failures reflect missing, unused, or insufficiently effective artifacts.
2.1 Fixed-Weight Agent Lineages
- Each lineage keeps model weights fixed while future instances inherit persistent artifacts; acting episodes start in fresh contexts without conversational history or temporary files.Training evidence informs proposed inventory updates, which must pass structural and executable checks before becoming persistent.Source: §2.1 · pp. 3–4
- Held-out ID and OOD scores are tracked across checkpoints to assess generalization beyond training interactions and transfer across opponent difficulty.The evaluation also reports full trajectories because persistent adaptation may not be monotone.Source: §2.1 · p. 4
2.2 Measuring Agent Plasticity
- Agent plasticity measures held-out performance gain per unit learning cost, with the headline measure summarizing acquisition efficiency up to estimated saturation.The saturation estimate uses a fitted monotone curve; if its fitted 90% rise is below 10 points, P_sat is set to zero.Source: §2.2 · pp. 4–5
2.3 Diagnosing the Improvement Loop
- Failure diagnosis separates absent covering artifacts, unused relevant artifacts, and failures that remain despite artifact use.The analysis also measures reuse among relevant decisions; its diagnostics are observational and cannot establish causal effectiveness or distinguish artifact quality from application.Source: §2.3 · p. 5
3 Experiments
The experiments test persistent learning by having agents revise reusable artifacts across checkpoints and evaluating performance on held-out interactions, including transfer and learning cost. Results vary across models and games, while artifact reuse and success when artifacts are used capture different parts of the improvement process.
3.1 Experimental Design
- At each checkpoint, agents played training and held-out games; reflection on training games updated persistent artifacts for later checkpoints.The setup included held-out ID and OOD evaluations, with separate actor episodes for games.Source: §3.1 · p. 6
3.2 Persistent Improvement Varies Across Models
- Hard-chess held-out ID scores rose from 37.5% to 73.3% for Claude Fable 5 and 25.0% to 66.4% for Claude Opus 5, while GPT-5.6 Sol reached 36.9% from near 0%.Several other models stayed near their initial performance or declined despite repeated opportunities to revise artifacts.Source: §3.2 · p. 6
- Held-out ID gains broadly accompanied training gains for the strongest improvers, but gains against stronger held-out OOD opponents were generally smaller or less consistent.The comparisons do not isolate environment experience from compute required for artifact development.Source: §3.2 · pp. 6–7
3.3 The Phenomenon Extends Across Games, but Is Task Dependent
- Across Go, Hex, and NetHack, improvement depended on both model and task: examples include large held-out gains, losses, and flat performance across lineages.Claude Fable 5 improved in Go, GPT-5.6 Sol improved in Hex, and Claude Opus 5.5 rose from about 2k to about 6k mean NetHack score.Source: §3.3 · p. 7
3.4 Acquisition Efficiency Differs from Endpoint Capability
- On the combined three-game curve, GPT-5.6 Sol’s estimated P sat was 298 percentage points per $1,000 of mean per-game learning cost, versus 233 for Claude Opus 5, 57 for Claude Fable 5, and 14 for Claude Opus 4.8.Claude Fable 5 reached the highest fitted performance level. GPT-5.5 was still improving slowly at run end, so its saturation and P sat cannot be established from these runs.Source: §3.4 · p. 7
3.5 Where Does the Improvement Loop Break?
- Across 19 model–game cells, held-out ID artifact reuse correlated with held-out ID score gain; low-reuse lineages often showed weak improvement.In pooled Hard and Easy chess, about 98% of GPT-5.6 Luna's relevant decisions lacked artifact use, compared with 2–6% for several strong improvers.Source: §3.5 · p. 8
- Reuse did not explain all performance differences: four models reused artifacts on 94–98% of relevant chess decisions yet attained substantially different scores.Among the strongest improvers, 83–99% of remaining failures occurred while a relevant artifact was in use, which does not guarantee that it generalizes or is applied well.Source: §3.5 · p. 8
- Reuse dynamics can diverge from task performance: reuse may rise before score peaks, failures can shift toward artifact-used decisions without improvement, and agents may later stop using unchanged artifacts.GPT-5.6 Sol and Claude Opus 4.8 reached 93–99% Hard-chess reuse by checkpoint 3, before held-out ID score peaks at checkpoints 12 and 5, respectively.Source: §3.5 · pp. 8–9
4 Discussion and Limitations
The discussion distinguishes persistent capability gains from context management and describes several limits on interpreting agent plasticity. Its failure analysis and experimental scope suggest different bottlenecks and leave some mechanisms unresolved.
4 Discussion and Limitations
- Plasticity measures whether mechanisms such as reflection and artifact reuse yield persistent held-out gains relative to learning cost, not context management alone.High artifact reuse can coexist with substantially different task performance, and the study does not isolate long-context instruction following’s contribution to artifact construction or application.Source: §4 · pp. 9–10
- After failures, Claude Fable 5 adjusted its chess engine’s draw valuation, while Claude Opus 5.5 restricted NetHack prayer to its controller; both fixes persisted at later checkpoints.The chess engine held a draw until Stockfish erred at checkpoint 18; the NetHack prayer healed at checkpoint 10.Source: Figure 8 · p. 10
- Failure patterns suggest distinct candidate bottlenecks: non-reuse of available artifacts for some agents, and artifact quality, generalization, or application for others.These interpretations are observational; artifact removal, replacement, or forced invocation could help distinguish the mechanisms.Source: §4 · p. 10
- The protocol isolates persistent adaptation through self-constructed artifacts but does not separate environment feedback’s value from the additional computation used to analyze experience and revise artifacts.Budget-matched reflection without feedback and controlled actor–artifact comparisons could disentangle these effects.Source: §4 · p. 10
- The environments range from controlled board games with well-defined feedback to NetHack’s long-horizon, partially observed setting, but separately trained lineages leave cross-task transfer untested.Finite evaluation sets also introduce checkpoint-level noise.Source: §4 · pp. 10–11
- The experiments do not establish cross-task transfer of learned artifacts or improvement strategies, and plasticity estimates depend on score headroom, learning horizon, saturation-fit assumptions, and provider pricing.The authors recommend reporting full learning curves alongside scalar plasticity measures rather than treating small differences as universal model rankings.Source: §4 · p. 11
- The experiments measure plasticity rather than optimize it; future work could test post-training objectives that reward constructing, retaining, and using knowledge to improve future behavior.The paper identifies suitable training objectives, feedback signals, and curricula as open questions.Source: §4 · p. 11
5 Conclusion
The conclusion presents agent plasticity as a measure of efficient improvement in held-out capability and reports that improvement varies across models and tasks. The appendix supplies additional experiment details, prompts, results, and qualitative examples.
5 Conclusion
- Agent plasticity measures how efficiently agents convert experience into persistent gains in held-out capability, revealing differences that endpoint performance alone obscures.Evaluating long-lived agents therefore requires measuring capability change through experience as well as capability at a fixed point in time.Source: §5 · p. 11
- Across models and tasks, plasticity varies substantially, persistent self-improvement is model- and task-dependent, and gains within the learning distribution need not fully transfer to harder settings.Artifact reuse is associated with improvement but does not guarantee success; distinct bottlenecks include retrieval, artifact quality, and application.Source: §5 · p. 11
Appendix
- The appendix provides supplementary experiment details, prompts, additional results, and qualitative examples, including checkpoint performance in Easy chess, Go, and Hex.Figure 9 complements Figure 3; game settings and Go and Hex learning curves are also provided in the appendices.Source: Appendix · p. 16, Figure 9 · p. 16
A Related Work
Related work spans continual learning, agent-learning benchmarks, persistent artifacts, self-evolving systems, and context management. This paper positions its contribution as measuring persistent improvement efficiency and tracing artifact-level failures under a common protocol.
A Related Work
- Continual-learning research studies acquiring tasks while preserving and reusing knowledge, while learning benchmarks examine feedback-driven improvement, sequences of problems, and long-term knowledge accumulation.The cited benchmarks also cover evolving agent configurations, long-term retrieval, and long-horizon environments.Source: A Related Work · pp. 16, 18
- Related language-agent benchmarks assess adaptation and generalization under task streams, while this work focuses on frozen frontier models using self-constructed artifacts and traces failures at the artifact level.The paper treats plasticity as complementary to retention, cross-task transfer, and adaptation under distributional change.Source: A Related Work · p. 17
- Persistent-artifact methods retain experience as reflections, reusable knowledge, workflows, memories, strategies, tools, or skills; this paper instead measures whether agents construct and reuse such artifacts to improve held-out performance.The paper’s focus is complementary to mechanisms for retaining and exploiting experience.Source: A Related Work · p. 17
- Self-evolving-agent and optimization approaches modify agent implementations, memory operations, skills, prompts, or programs, whereas this study fixes the improvement interface and measures persistent adaptation efficiency throughout learning.The same model acts and reflects within each lineage.Source: A Related Work · pp. 17–18
- Context-management evaluations concern retaining, selecting, or using information, whereas plasticity concerns persistent capability gains relative to learning cost; the experiments do not isolate long-context instruction following’s role in artifact construction or application.Agents also construct and manage artifacts such as executable tools that need not reside in token context.Source: A Related Work · p. 18
- The framework combines inspectable self-constructed artifacts, cost-normalized held-out learning trajectories, and artifact-level failure diagnostics under a common protocol.The diagnostics distinguish absent artifacts, available-but-unused artifacts, and failures that persist despite reuse.Source: Table 1 · p. 17, A Related Work · p. 18
- The paper places its fixed-weight, persistent-artifact setting between iterative test-time refinement that need not persist and adaptation through parameter updates.Its use of plasticity is a system-level property relative to a learning protocol, and the present measurements isolate improvement through persistent artifacts.Source: A Related Work · p. 18
B Experimental Setup
The experimental setup evaluates four environments with different task distributions, feedback, and observation structures. Only training experience reaches reflection; held-out evaluation and separate offline failure attribution remain outside the learning loop.
B Experimental Setup
- The four-environment setup distinguishes task distributions, opponent difficulty, evaluation-set sizes, feedback exposed to reflection, and offline annotations; difficulty scales are game-specific, not a common Elo scale.Only training trajectories reach reflection, and held-out trajectories and scores never do.Source: B Experimental Setup · p. 19, Table 2 · p. 19
- Chess, Go, and Hex are fully observed alternating-turn board games in which actors inspect state and legal actions, may use a reversible analysis board, and commit one move before the engine responds.Chess starts from the standard position; Go changes deterministic training seeds while keeping held-out seeds fixed; Hex uses fixed banks and deterministic openings.Source: B Experimental Setup · p. 20
- NetHack is a long-horizon, partially observed single-agent environment with a documented interface exposing public observations and actions; each lineage evaluates ten fresh seeded Valkyrie games per checkpoint.Games are bounded by 50,000 primitive actions and two hours of active play, across checkpoints 0–40.Source: B Experimental Setup · p. 20
- Feedback differs across environments: Chess supplies expert move-value estimates, Go and Hex give lighter diagnostics, and NetHack provides execution telemetry without an optimal-action oracle.Stronger Go and Hex adjudications are reserved for offline failure analysis and cannot improve evaluated agents.Source: B Experimental Setup · p. 20
- Reflection receives training trajectories, learning-time annotations, traces, and permitted per-episode state, while held-out splits and offline artifact/failure attribution remain evaluation-only.Held-out trajectories and scores are not supplied to reflection.Source: B Experimental Setup · p. 20
C NetHack Learning Curves
Across NetHack checkpoints 0–40, Claude Opus 5.5 is the only lineage with a significant fitted performance increase, though its later scores vary substantially. Its early strategy drove a large gain, while checkpoint comparisons also reflect game difficulty and baseline differences.
C NetHack Learning Curves
- The six lineages played the same ten new games at each checkpoint, but checkpoint-to-checkpoint changes also reflect game difficulty.The series covers checkpoints 0–40; Figure 10 plots mean scores, while Figure 11 shows median score and mean deepest dungeon level.Source: C NetHack Learning Curves · p. 21, Figure 10 · p. 21, Figure 11 · p. 21
- 68.95 normalized points was Claude Opus 5.5’s fitted total rise (p = 8.23 × 10−7); no other model had a significant performance increase.Checkpoint 11 was the first observed checkpoint reaching 90% of the fitted rise; its measured prior learning cost was $1,072.73, yielding P_sat = 61.75 points per $1,000.Source: C NetHack Learning Curves · p. 21
- Claude Opus 5.5’s checkpoint-10 mean was 9,015, but its later checkpoint means ranged from 3,518 to 16,689, with 5,831 over checkpoints 36–40.The early strategy was to find a digging tool or, without one, kill monsters to gain experience; the checkpoint-10 jump came mostly from three games following the second rule.Source: C NetHack Learning Curves · p. 22
- Claude Opus 5.5’s checkpoint-10 lead is not explained by greater tool code or tool use, and its three best games account for 69% of its total.It already scored 2.6–25× the other lineages at checkpoint 0; the other lineages also used earlier harness versions.Source: Table 3 · p. 22
D NetHack Examples
The NetHack examples show how reusable safeguards can improve play, but also how agents may bypass them or fail to interpret the environment. One agent’s travel command overcame a door-planning failure that stalled another.
D NetHack Examples
- At checkpoint 1, Claude Opus 5.5 prayed through raw keystrokes and died; after adding a “Pray only via nhctl pray” rule, it waited for the controller at checkpoint 10 and healed.Later agents still sometimes bypassed the prayer check and died.Source: Figure 12 · p. 23
- GPT-5.6 Sol treated an open door displayed as a wall as impassable, kicked it, and ended on dungeon level 1; Claude Opus 5.5 used travel to pass it and reached the stairs.The Claude Opus 5.5 game scored 15,718, mostly earned later; its notes guessed where the stairs were based on recurring layouts.Source: Figure 13 · p. 24
E Chess Examples
The chess examples trace both successful and failed adaptation: agents add engines and tactical rules, but some artifacts remain narrow or are bypassed. Resource-saving behavior can also degrade move quality.
E Chess Examples
- At checkpoint 18, GPT-5.6 Sol’s engine played every move in a held-out game, ending in checkmate on move 42; at checkpoint 0, its own move selection lost a pawn and knight.The agent had added the engine to its library at checkpoint 1.Source: Figure 14 · p. 25
- A checkpoint-15 avoid-list entry prevented GPT-5.6 Sol’s engine from repeating 3.Qg4 in the same held-out position at checkpoint 16.Removing the entry made later engines play 3.Qg4 again, so the fix helped only when that position recurred.Source: Figure 15 · p. 26
- Claude Fable 5’s engine increasingly valued draws near the 300-ply limit, keeping a held-out game going until Stockfish erred and lost by checkmate.The opponent’s error produced the win; the draw-valuing rule kept the game going until then.Source: Figure 16 · p. 27
- Claude Opus 5’s engine sought draws when losing, first by repetition and later by stalemate after giving away its queen.In the held-out game, the stalemate produced a draw rather than a loss.Source: Figure 17 · p. 28
- GPT-5.5’s checks against early Nxe5 did not prevent the move because its opening book selected it before the checks ran.At checkpoint 20, the held-out blunder cost 263 cp, and the game was lost despite rules accumulating from checkpoint 5.Source: Figure 18 · p. 29
- GPT-5.6 Luna’s instruction to choose the first legal move raised that behavior to 0.65 of held-out moves at checkpoint 20, including moves that lost a knight or allowed checkmate.The change aimed to avoid exhausting 64 model calls per game, but some games still ran out of calls.Source: Figure 19 · p. 30
F Additional Results
The appendix plots performance against cumulative learning tokens and learning cost in Chess, Go, and Hex. These views compare how score trajectories change as learning resources accumulate.
F.1 Performance Versus Learning Tokens
- Figures 20–22 plot score against cumulative learning tokens for Chess, Go, and Hex, with three-checkpoint averages shown.For Chess, learning tokens count model calls spent playing training games and reflecting to write the next checkpoint; held-out games are excluded.Source: Figure 20 · p. 31, Figure 21 · p. 31, Figure 22 · p. 31
F.2 Performance Versus Learning Cost
- Figures 23–25 plot score against cumulative learning cost in dollars for Chess, Go, and Hex.The Chess figure uses the learning-cost measure defined in Section 2.2; the Go and Hex plots show three-checkpoint averages.Source: Figure 23 · p. 32, Figure 24 · p. 32, Figure 25 · p. 32
G Failure and Reuse Analysis Procedure
The procedure uses an offline analyzer to assign failure and artifact-reuse labels to games. It distinguishes missing coverage, non-use, and failure despite use, with labels checked against evidence and logs.
G Failure and Reuse Analysis Procedure
- Failures are classified as Fabsent when no artifact covers them, Fmissed when a covering artifact is unused, and Fused when failure persists despite artifact use.A decision counts as reused if any relevant artifact was used.Source: G Failure and Reuse Analysis Procedure · p. 33
- Gemini 3.1 Pro analyzes each game offline through six stages, using the game record, artifacts, and game rules.Analysis covers training and held-out ID games at checkpoints 0–20.Source: Table 4 · p. 33, G Failure and Reuse Analysis Procedure · p. 33
- Artifact-use claims are retained only when logs show the artifact was run or read; analyzer requests also include engine evaluations agents never saw.The analyzer is instructed to treat records as data and use only supplied evidence and game rules.Source: Table 4 · p. 33
H Failure and Reuse Diagnostics Across Games
The appendix tracks failure and artifact reuse across games, checkpoints, and evaluation splits, then documents the prompts and inventory rules used by acting and reflection agents. The diagnostics distinguish failures with no covering artifact, failures where an artifact was unused, and failures that remained despite use.
H Failure and Reuse Diagnostics Across Games
- Reuse outcomes are tracked by checkpoint and game; artifact-conditioned success rates are associations, not causal effects.Relevant decisions are those where at least one saved artifact applies to choosing or playing the move.Source: Figure 27 · p. 34, Figure 36 · p. 38, Figure 37 · p. 39
- Failure diagnostics distinguish absent coverage, an available but unused artifact, and failure despite artifact use across training and held-out ID games.Figure 30 reports fractions within each model, game, and split; its counts are verified versus identified failures.Source: Figure 28 · p. 34, Figure 30 · p. 35, Figure 29 · p. 35
- Late chess failures are mostly unused-artifact cases for Gemini 3.1 Pro and GPT-5.6 Luna, but mostly failures despite use for Claude Opus 5, Claude Opus 4.8, and GPT-5.6 Sol.Claude Fable 5 and Claude Opus 5 make fewer failures late; GPT-5.5 shifts toward non-use.Source: Figure 31 · p. 36
I Actor and Reflection Prompts
- The actor and reflection prompts are standardized across models, with game-specific rules, APIs, call budgets, and tool limits.Most non-Claude Fable 5 Go and Hex runs use an inference gateway that appends a tool-calling fallback instruction.Source: I Actor and Reflection Prompts · p. 41, §I.3 · p. 43
I.1 System prompt (all games, actor and reflection)
- The shared system prompt directs agents to improve scores by building infrastructure from interactions, feedback, rewards, and failures.Agents may create or revise tools, persistent memory, and strategies, and remove unneeded inventory.Source: §I.1 · p. 41
I.2 Actor prompts
- Actor prompts provide persistent infrastructure and game APIs, while imposing game-specific action and model-call limits.Chess allows at most 64 model calls; Go and Hex allow 32, with distinct rules and action interfaces.Source: §I.2 · pp. 41–42
I.3 Reflection prompt
- Reflection sees training-game feedback but not held-out games, and must maintain failure-mode and artifact inventories linked to trajectories and addressed failures.Failure-mode IDs are reused for the same causal problem; artifact IDs remain stable for revised capabilities, and inventory files are validated.Source: §I.3 · pp. 42–43
- Reflection can retry rejected inventory edits, consulting prior retry history and logs to resolve validation errors rather than repeat them.Retries are capped at six attempts in chess and three in Go and Hex.Source: §I.3 · p. 43