PaperlayerView exact revision on arXiv

Source-linked AI summary

From Solo to Social Learning: Characterizing Recursive Social Improvement in LLMs

Kunal Jha, Max Kleiman-Weiner, Natasha Jaques

arXiv:2609.38516v1cs.MAcs.AIcs.LG

TL;DR

Self-improvement methods usually optimize one system at a time, while many multi-agent systems pursue a shared goal; the paper asks whether self-interested LLM agents can learn from one another. It studies agents that revise and exchange skills while allocating a shared token budget to learning and acting. Peer access changes learning and can improve efficiency, but the LLM populations do not outperform independent learners at matched cost.

  • Problem

    Prior instruction-improvement methods typically optimize one system at a time, while multi-agent work often coordinates models toward shared goals, leaving the self-interested peer-learning setting as the question studied here.

    Source: §1 · p. 1

  • Method

    The study compares populations that revise, copy, or skip skill learning and act under one token budget with independent learners at equal total cost.

    Source: §1 · p. 2, §3 · p. 4

  • Results

    LLM agents’ peer access changes learning speed and cost, but neither tested model shows a clear advantage over independent learners at matched learning tokens.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    LLMs can build on peers’ skills and improve learning efficiency, but the reported populations are not yet more effective than equal-cost independent learners.

    Source: §1 · p. 2

    The study covers three models in bandit experiments and two models on one instruction-following benchmark, so findings may not generalize to other models or tasks.

    Source: §6 · p. 9

Abstract

from arXiv · show

Large language models (LLMs) can now improve themselves by revising the instructions they follow, and LLM agents are increasingly orchestrated to work together on complex problems. However, self-improvement methods typically optimize one system at a time, and multi-agent frameworks often have every model work toward a shared goal. We ask a different question. When each agent pursues its own reward, can self-improving LLMs learn from one another well enough to improve the whole population? We call this capability recursive social improvement. We study populations that revise skill files and choose whether, when, and whom to copy from. Independent search, learning from peers, and acting all share one token budget. In controlled environments, established social-learning algorithms benefit from peers, but three LLMs do not. They earn less reward per token than solo learners, and explore too narrowly or run out of tokens before acting. We then let the models write and revise their own skills. Observing peers changes how they improve, helping one model find useful skills sooner and another spend less on private search. Neither, however, outperforms independent learners at the same cost. Skills are copied, revised, and passed on, so one discovery can seed further search. Yet these exchanges concentrate the population around fewer independent discoveries. Together, these results show that LLMs can make learning more efficient by copying from peers, but not yet more effective.

1 INTRODUCTION

The introduction asks whether self-interested LLM agents can learn from peers while improving their own procedures, rather than coordinating toward a shared task. It defines recursive social improvement and reports that peer learning changes how agents learn but does not beat independent learners at matched cost.

1 INTRODUCTION

  • Prior self-improvement methods typically optimize one system at a time, while much multi-agent work coordinates models toward a shared task; this paper studies agents pursuing separate rewards while seeing peers’ work.Skill files can be revised and shared without changing model weights.Source: §1 · p. 1
  • Recursive social improvement is agents improving their own procedures, learning from others’ improvements, and revising what they acquire so one discovery can seed another’s search.The setting raises a tension: social learning needs independent search, yet each agent would prefer another to pay for it.Source: §1 · p. 2
  • The study compares social populations with independent learners at equal total token cost, counting search behind copied skills, and tests both a controlled environment and instruction-following tasks.Agents choose whether to search privately, observe a peer, or act from the same token budget.Source: §1 · p. 2
  • Every tested LLM was 30–52% less efficient when observing peers than solo populations of the same model, and neither model showed a clear advantage over independent learners at matched learning tokens.GLM-5.3-Flash averaged 1.3 percentage points higher held-out accuracy across learning, while GPT-OSS-120B used a third fewer learning tokens; skills in most agents traced back to one or two first revisions.Source: §1 · p. 2
  • The authors conclude that LLM agents build on peers’ skills but do not outperform equal-cost independent learners, and suggest studying training or incentives for balancing exploration and observation.The proposed capability is intended to matter as self-interested agents improve side by side.Source: §1 · p. 2

2 RELATED WORK

The related-work section distinguishes this study from single-system instruction improvement and shared-goal multi-agent collaboration. It brings social-learning and costly-information perspectives to self-improving agents that choose how to divide computation between peer learning and acting.

Self-improving LLM agents

  • Prior self-improvement methods improve one system at a time, whereas this work asks what happens when many such systems improve side by side and see one another’s work.These methods include prompt optimization, reflection, evolutionary search, and reusable skill libraries.Source: §2 · p. 3

Multi-agent collaboration and swarms

  • Unlike multi-agent systems designed for collaboration toward a common objective, this setting has agents maximize individual rewards and choose whether to reuse peers’ work.Any population-level benefit must emerge from those individual choices.Source: §2 · p. 3

Social learning, cultural evolution, and games

  • Social-learning research studies the tension between producing and reusing information, while tournaments and bandit algorithms formalize discovery, observation, and exploitation as repeated choices.This paper applies that tradition to agents sharing revisable procedures and dividing one token budget between learning and acting.Source: §2 · p. 3

Costly information and the allocation of computation

  • Research on costly information considers how social information affects private information acquisition and whom agents choose to observe; here, peer selection and skill interpretation compete with acting for tokens.The study asks how those choices affect individual performance and the population’s.Source: §2 · p. 3

3 RECURSIVE SOCIAL IMPROVEMENT AS A REINFORCEMENT LEARNING

The paper formulates social improvement as agents choosing learning actions and task execution under a shared, finite token budget. It distinguishes performance per token from social advantage at matched cost, requiring recursive transmission and positive matched-cost advantage for recursive social improvement.

Population and task setup

  • Agents solve their own task distributions using skills from their libraries, with policies conditioning on histories of actions, rewards, and observations.Skills range from bandit arms to written skill files, and experimental policies are fixed-weight LLMs or algorithmic baselines.Source: §3 · p. 3

A two-stage action space

  • Each agent first revises a skill, observes a chosen peer’s previously executed skill and reward, or skips learning, then chooses a library skill to execute.Observation costs tokens rather than a round’s payoff, making its cost depend on the agent’s reasoning; peers’ ongoing learning also makes observation value change over time.Source: §3 · pp. 3–4

Learning competes with acting

  • Each round gives an agent the same allowance for choosing and carrying out learning, choosing a skill, and executing it, so lengthy decisions can leave too little budget to act.Population cost counts tokens spent by all agents, including discovery search; a copier pays to acquire and use a skill, not for its original search.Source: §3 · p. 4

Measuring improvement at matched cost

  • The framework evaluates population performance at cost C using the last round before cumulative spending exceeds C, and compares social and solo populations at that matched cost.A performance-per-token ratio can rise simply because a social population stops searching sooner, so it does not by itself establish greater effectiveness.Source: §3 · p. 4

What counts as recursive social improvement

  • Recursive social improvement requires both recursive transmission—a revised skill copied and revised again—and positive social advantage over independent learners at matched total cost.This requirement distinguishes population-level improvement from parallel search that merely shares discoveries cheaply.Source: §3 · p. 4

4 UNDERSTANDING LLM SOCIAL LEARNING IN A CONTROLLED

The controlled bandit tests whether LLMs benefit from observing peers when observation, private search, and acting share a token budget. Algorithmic social learning benefits from peers, but LLM populations show no reliable advantage over solo learners and often explore less or spend tokens before acting.

An algorithmic social learner benefits from observing peers (RQ1).

  • Hierarchical UCB earns 25.68 ± 2.87 more cumulative reward than solo UCB over 100 rounds, mainly by finding good arms sooner.The gain also holds across 100 simulated environments and when payoffs change every 25 rounds, but weakens when they change every 10 rounds.Source: §4 · p. 5, Figure 2 · p. 5

LLM social learners show no clear advantage over solo learners at matched cost and use tokens less efficiently (RQ1).

  • At matched completion-token cost, social-minus-solo reward differences are +0.14 ± 0.33 for Qwen, +0.37 ± 0.34 for Ministral, and −1.24 ± 0.45 for GPT-OSS, none significant after correction.All three models earn less reward per token with observation: drops of 29.6±13.4%, 41.8±11.6%, and 51.9 ± 7.8%, respectively.Source: §4 · p. 5, Figure 2 · p. 5

Copying helps the individual but narrows the population’s search (RQ2).

  • Social populations explore fewer arms, while lengthy decisions about whether and whom to observe can leave agents without a completed pull.Qwen completes pulls in 81.0% of social rounds versus 98.1% of solo rounds; GPT-OSS social populations quickly use fewer distinct arms than solo populations.Source: §4 · p. 6, Figure 3 · p. 6

Fixing exploration and execution does not make social learners outperform solo learners (RQ2).

  • Protected execution and independent search fix their targeted problems, but neither makes social learners reliably outperform equally treated solo learners.Protected execution reserves 2,048 of 4,096 tokens for a pull; independent search blocks observation in 15% of assigned rounds and requires an untried, unobserved arm when available.Source: §4 · p. 6, Figure 4 · p. 6

Effective social learning requires more than access to useful information (RQ2).

  • Effective social learning also requires balancing peer information against private discoveries; the study finds no reliable social advantage across cumulative reward, reward per token, or matched-cost reward.Similar patterns appear in an effectively infinite-arm bandit and on reasoning tasks with fixed skill files.Source: §4 · p. 6

5 CAN SELF-IMPROVING LLM AGENTS BENEFIT FROM ONE ANOTHER?

The paper tests whether self-improving agents can benefit from peers when they revise and exchange skill files, and whether simple controllers can choose whom to observe. At equal learning-token cost, neither LLM shows a clear accuracy advantage; social access changes when models improve and how much they spend, while populations build on fewer independent discoveries.

Setup and learning procedure

  • Agents revise, evaluate, keep, or observe skills, then execute a library skill; copied skills can be revised and passed on, with model weights and decision prompts held fixed.The evaluation uses IFBench, six five-agent populations per condition, 49 learning rounds, and 200 held-out examples.Source: §5 · p. 7

Neither LLM shows a clear accuracy advantage at matched learning computation (RQ3).

  • At equal population-wide learning tokens, social held-out accuracy is +0.14±0.35 percentage points for GPT-OSS and +0.98 ± 0.85 for GLM versus solo, with neither difference statistically significant.The authors therefore find that neither model meets their criterion for recursive social improvement.Source: Table 1 · p. 7, §5 · p. 8

GLM improves sooner with social access, while GPT-OSS spends less (RQ3).

  • GLM’s social populations average 1.31 ± 0.61 points higher held-out accuracy during learning, while GPT-OSS shows no accuracy improvement and uses 17.34 ± 2.08 million fewer learning tokens.GLM’s final accuracy is close to solo; GPT-OSS’s cost savings partly reflect avoiding the 100-example evaluation required for each new revision.Source: Figure 5 · p. 7, §5 · p. 8

Spreading observations across learning works better than observing early (RQ3).

  • With four observations per agent, distributed observation beats early observation for GLM by 2.42 ± 0.46 points and beats solo by 3.00 ± 1.13 points; GPT-OSS’s similar gains are not significant.Under distributed observation, 95.83% of GLM’s copied skills differ from the shared initial file, versus 79.17% with early observation.Source: Figure 6 · p. 8, §5 · p. 9

Skills are revised and passed on, but they come from fewer independent discoveries (RQ3).

  • Recursive transmission occurs, but social populations build on fewer independent skill lineages: GPT-OSS averages one lineage versus 4.83 ± 0.17 under solo learning, and GLM 2.33 ± 0.21 versus five.At the final round, 83.3 ± 16.7% of GPT-OSS skills and 56.7 ± 6.1% of GLM skills show recursive transmission.Source: §5 · p. 9

Algorithmic social controllers reduce search costs without a clear benefit from adaptive source choice (RQ4).

  • UCB and uniform controllers use 76–78% fewer learning tokens than full solo OpenEvolve while final accuracy stays within 1.03 points; adaptive source choice has no clear advantage over uniform choice.The figure reports final held-out accuracy and end-to-end token use, including held-out evaluation, per five-agent population.Source: Figure 7 · p. 8, §5 · p. 9

6 DISCUSSION

LLM agents learn from peers but do not outperform independent learners at equal cost; the discussion identifies peer-selection decisions and study scope as important boundaries.

6 DISCUSSION

  • An algorithmic social learner benefits from peer observation, but none of the tested LLMs outperforms independent learners at equal cost.The LLMs either explore too little or spend tokens before acting; fixing either issue alone does not close the gap.Source: §6 · p. 9
  • Peer observation changes learning behavior: GLM reaches higher accuracy sooner, while GPT-OSS spends less and becomes more efficient, but neither becomes more effective.Agents revise and pass on copied skills; the remaining gap concerns when and from whom they learn.Source: §6 · p. 9
  • Findings are bounded by three bandit models and two models on one instruction-following benchmark, with peers always listed in the same order.Other constraints include same-distribution tasks, one model per population, latest-observation retention, and fixed decision prompts.Source: §6 · p. 9

AI USE

Generative AI assisted with implementation, analysis, reference-finding, and manuscript work; the authors retained responsibility for the research design and final content.

AI USE

  • Generative AI assisted with experimental and analysis code, results processing, figure production, reference-finding, and drafting and editing portions of the manuscript.The assistance included synthetic bandit simulations and the OpenEvolve port.Source: AI USE · p. 10
  • The authors originated the research questions, selected benchmarks and models, designed conditions and controls, interpreted results, and wrote and edited the draft.Source: AI USE · p. 10
  • AI was not used for theoretical results or proofs, translation, qualitative analysis, or survey, interview, and transcription tasks.The authors manually verified suggested references, reviewed and tested AI-assisted code, checked figures against seed-level results, and manually revised the manuscript.Source: AI USE · p. 10

REPRODUCIBILITY STATEMENT

The authors describe the paper’s experimental documentation and planned release of code, configurations, and seed-level summaries.

REPRODUCIBILITY STATEMENT

  • Sections 3–5 describe the formalization, environments, models, comparison conditions, data splits, and evaluation metrics.Source: REPRODUCIBILITY STATEMENT · p. 10
  • The appendix specifies bandit distributions and algorithmic baselines, skill-evolution budgets and accounting, uncertainty procedures, implementation checks, and additional results.Source: REPRODUCIBILITY STATEMENT · p. 10
  • The authors plan to release the experimental and analysis code, configurations, and seed-level summaries, and preserve matched seeds across primary comparisons.Uncertainty is estimated after aggregating agents within independent populations.Source: REPRODUCIBILITY STATEMENT · p. 10

A FINITE-BANDIT IMPLEMENTATION AND ADDITIONAL ANALYSES

Additional analyses compare reward and regret, probe social-learning mechanisms, and document the finite-bandit setting and algorithmic baselines.

Finite-bandit environment

  • The stationary bandit draws 25 arm means independently from an exponential distribution with mean one; the best arm is well separated from the median on average, though its gap over second-best is sometimes smaller.Matched seed-keyed Gaussian counterfactual rewards let policies face the same potential outcome for a given pull; peers reveal only completed previous-round actions and payoffs.Source: A FINITE-BANDIT IMPLEMENTATION AND… · p. 15, Figure 8 · p. 16

Matched-cost and precision checks

  • The sensitivity checks retain the primary social-minus-solo directions: last-round differences are +0.33 ± 0.25 for Qwen, +0.32 ± 0.36 for Ministral, and −1.30 ± 0.45 for GPT-OSS.Using prompt-plus-completion tokens gives +0.16±0.30, +0.38±0.35, and −1.12±0.48, respectively; the fixed 100-seed UCB extension improves precision for the paired algorithmic contrast.Source: A FINITE-BANDIT IMPLEMENTATION AND… · pp. 16–17

Regret analysis

  • At round 100, solo UCB has 0.25 expected regret per pull, hierarchical UCB 0.11, and the oracle zero; GPT-OSS accumulates more regret socially than alone.Qwen and Ministral have overlapping regret trajectories; reward and regret use the same actions and oracle means.Source: A FINITE-BANDIT IMPLEMENTATION AND… · p. 17, Figure 12 · p. 18

Population dynamics

  • GPT-OSS social populations rapidly use a narrower set of arms than solo populations, while Qwen also uses fewer distinct arms socially for much of the run.These trajectories are descriptive; the randomized independent-search intervention supplies the causal test.Source: A FINITE-BANDIT IMPLEMENTATION AND… · p. 17, Figure 14 · p. 18

Completion-aware outcomes

Independent-search intervention

  • IS raises mean reward by 1.031 ± 0.154 in solo and 0.327 ± 0.231 in social populations, yielding a social-minus-solo interaction of −0.704 ± 0.287.Completion falls by 3.56 percentage points in solo and 12.10 in social populations; these randomized effects identify a tradeoff but do not isolate mediation through completion versus arm choice.Source: A FINITE-BANDIT IMPLEMENTATION AND… · p. 19, Figure 17 · p. 20

Hierarchical source selection

  • Hierarchical source selection uses observed source payoffs and gives an agent a modest initial prior advantage over peers; changing peer skills mean standard stationary-arm UCB guarantees do not apply.Source scores are training scores, not estimates of marginal gains from copying or revising.Source: A FINITE-BANDIT IMPLEMENTATION AND… · p. 21

Discounted explore–observe–exploit

  • DEOE explores independently with probability 1/√t + 1, otherwise observing after surprising payoffs, 20 unobserved rounds, or the same decaying exploration probability, then exploiting its highest discounted estimate.Discounting is unnecessary for stationary means in an ideal estimator, but the policy keeps one rule across stationary and changing conditions.Source: A FINITE-BANDIT IMPLEMENTATION AND… · p. 21

B SOCIAL LEARNING WITH AN EFFECTIVELY INFINITE OPTION SET

In continuous-bandit settings, social access reduces independent discoveries, but its reward effect depends on the model and payoff stability. A hand-designed policy benefits from payoff-visible social information, while faster payoff changes weaken the LLM benefit.

B SOCIAL LEARNING WITH AN EFFECTIVELY INFINITE OPTION SET

  • Ten agents interact for 100 rounds in the primary continuous-bandit landscape, with 4,096 expiring tokens per round shared across innovation, observation, and reward-earning action choices.INNOVATE samples a coordinate and reveals informational payoff; OBSERVE reveals a peer’s recent coordinate; only PULL earns scored reward.Source: B SOCIAL LEARNING WITH AN EFFECTIVELY… · p. 21
  • Social access sharply reduces independent arm discoveries for all three models, while only GPT-OSS-20B shows a clear stationary-reward benefit; Qwen and Ministral do not.The reduced discovery does not necessarily lower short-run performance.Source: B SOCIAL LEARNING WITH AN EFFECTIVELY… · p. 21, Figure 22 · p. 23
  • GPT-OSS-20B’s social-minus-solo reward is +0.820±0.471 when stationary, +0.049±0.076 with 25-round changes, and −0.034 ± 0.065 with 10-round changes.Continuous-bandit regret is measured against the best value on a dense reference grid, so it is approximate rather than an exact infinite-arm optimum.Source: B SOCIAL LEARNING WITH AN EFFECTIVELY… · p. 24, Figure 23 · p. 24, Figure 26 · p. 25
  • Social populations can reduce independent discoveries early even when short-run reward effects are small or positive.Reward trajectories and independent-discovery trajectories show when the differences emerge.Source: B SOCIAL LEARNING WITH AN EFFECTIVELY… · p. 24, Figure 24 · p. 24
  • DEOE’s payoff-visible social population gains +0.331 ± 0.076 reward per round over solo in the stationary length-scale-0.15 environment.This is a policy-level benefit under DEOE’s cost model, not evidence that every reduction in innovation is optimal.Source: B SOCIAL LEARNING WITH AN EFFECTIVELY… · p. 24, Figure 27 · p. 25

C SOCIAL LEARNING WITH FIXED REUSABLE SKILLS

With fixed reusable skills, social redistribution helps when it grants access unavailable to solo agents, but most of that advantage disappears when access is equalized. LLMs transmit and revise skills across agents, while deployed lineages concentrate and accuracy estimates remain sensitive to test-item sampling.

C SOCIAL LEARNING WITH FIXED REUSABLE SKILLS

  • The fixed-skill benchmark uses eight Markdown procedures for scheduling 16 deadline-and-profit jobs, scoring selected profit relative to the optimal profit.Invalid schedules receive zero reward.Source: C SOCIAL LEARNING WITH FIXED REUSABLE… · p. 26
  • Most of the original social advantage disappears when solo and social agents can independently acquire any of the eight skills.Redistribution was useful when it was the only route to better procedures, but provides little average value once both conditions can reach the same menu.Source: C SOCIAL LEARNING WITH FIXED REUSABLE… · p. 26
  • The DEOE+LLM hybrid improves five of six model-by-access cells, with its largest increase for Ministral social execution.Because DEOE selection also protects execution tokens, this is not a pure selector effect.Source: C SOCIAL LEARNING WITH FIXED REUSABLE… · p. 26, Figure 30 · p. 27

D.1 REVISION AND TRANSMISSION IN DEPLOYED SKILL LINEAGES

  • Copying followed by further revision appears in 83.3 ± 16.7% of GPT-OSS and 56.7 ± 6.1% of GLM final social deployments.This requires transmission of a non-initial revision and another private revision later in its lineage; the final user need not be the revising agent. Lineages count first-acquisition ancestry among deployed skills, not semantic diversity.Source: §D.1 · p. 27, Figure 31 · p. 28
  • GPT-OSS deploys one founding lineage socially versus 4.83 ± 0.17 in solo; GLM deploys 2.33 ± 0.21 socially versus five in solo.Social exchange supports revision chains while concentrating the ancestry of deployed skills.Source: §D.1 · p. 27, Figure 31 · p. 28
  • Mean proposed revision changes are below recorded parent fitness in both ancestry groups, but this does not preclude improvement of the selected skill.Parent scores were not freshly reevaluated and ancestry was not randomized, so these contrasts do not establish the causal benefit of copying the same starting file.Source: §D.1 · p. 27, Figure 32 · p. 28

D.2 UNCERTAINTY ACROSS POPULATIONS AND TEST ITEMS

  • Uncertainty checks resample the existing 200 test tasks and population seeds, not a new task distribution or new stochastic executions.The bootstrap preserves matching across conditions, seeds, and checkpoints.Source: §D.2 · p. 27
  • GLM’s social-minus-solo trajectory advantage is 1.31 percentage points (joint SE 0.61); final contrasts are −0.58 points for GPT-OSS and +0.95 for GLM (joint SE 0.88 and 1.15).The point estimate is unchanged, but consistent population seeds do not eliminate uncertainty from a finite test set.Source: §D.2 · p. 29, Figure 33 · p. 29

D.3 PRIVATE IMPROVEMENT AND LEARNING TRAJECTORIES

The corrected population experiments show that agents can improve their starting skills, but at matched learning-token expenditure LLM-social populations do not significantly outperform LLM-solo. The section distinguishes performance from resource use and explains how observing peers competes with private search.

D.3 PRIVATE IMPROVEMENT AND LEARNING TRAJECTORIES

  • Corrected population runs gain whole-answer accuracy over their own starting skills across the controllers shown, with corresponding gains on individual constraints.This is a within-run comparison, not a comparison with a separately executed frozen condition.Source: §D.3 · pp. 29–30, Figure 35 · p. 30
  • 9.02 ± 0.55 versus 2.03 ± 0.42 points for GPT-OSS, and 14.25 ± 1.11 versus 7.95 ± 0.84 for GLM: LLM-social training constraint gains exceed held-out gains.Reused training scores and different per-example token caps mean the gap is not a clean estimate of overfitting alone.Source: Figure 36 · p. 30, §D.3 · p. 30
  • GLM’s LLM-social trajectory remains higher on average after subtracting each run’s initial accuracy, by 1.03 ± 0.42 percentage points.This sensitivity check addresses starting offsets; the trajectory integral includes initial performance differences and is distinct from the matched-token comparison.Source: §D.3 · pp. 30–31

E.1 COMPARISON AT MATCHED LEARNING TOKENS

  • +0.14 ± 0.35 percentage points for GPT-OSS and +0.98 ± 0.85 for GLM: neither LLM-social advantage over LLM-solo at matched learning tokens is statistically significant.The comparison interpolates each seed’s social and solo runs at their common token endpoint; tokens include the private search behind copied skills.Source: §E.1 · p. 35

E.2 WHY COMPARE PERFORMANCE AT MATCHED COMPUTATION?

  • Reward per token and performance at equal expenditure answer different questions; the paper counts the population’s search behind copied skills rather than treating discoveries as free.Recursive social improvement asks whether agents can build on copied and revised discoveries to reach better performance than equally resourced solo learners.Source: §E.2 · pp. 35–36
  • 25.68 ± 2.87 more cumulative reward per agent over 100 rounds: hierarchical social UCB benefits from peers without additional LLM inference tokens.Solo UCB and hierarchical social UCB both use zero tokens under the paper’s accounting, so the comparison uses cumulative reward directly.Source: §E.2 · p. 36

E.3 TOKEN ALLOWANCES AND REALIZED USE

  • No completed agent-round exhausts the aggregate token allowance, but individual calls can still hit their caps or be constrained by reservations.The allowance permits full-batch search and is not intended to reproduce the tight finite-bandit budget.Source: §E.3 · p. 36

E.4 READING LEARNING COSTS

The section reports performance alongside cumulative tokens and API expense, distinguishing learning-only resources from end-to-end costs. Its comparisons describe realized policies and conditional skill-use patterns, not randomized equal-budget effects.

E.4 READING LEARNING COSTS

  • Total tokens count prompts and completions together; provider-reported USD complements token counts because providers price those components differently.Completion-only accounting can mislead when fewer generated tokens are offset by longer prompts.Source: §E.4 · pp. 36–37
  • The performance–spending plots follow saved checkpoints across rounds and compare social with solo within each model; separate vertical scales preclude comparing slopes across models.The plots characterize realized policies, not randomized experiments at a shared dollar budget or optimal performance frontiers.Source: §E.4 · pp. 37–38, Figure 50 · p. 37
  • Learning-cost measures include calls that affect evolving skills, while end-to-end measures add held-out generations; hidden evaluation is excluded from learning costs and included end to end.Reported API charges are not local compute costs, and unresolved usage reservations are not treated as free.Source: §E.4 · pp. 37–38, Figure 51 · p. 38
  • 1,694 ± 104 versus 4,473 ± 1,525 execution tokens for GPT-OSS early in learning, and 872 ± 253 versus 3,764 ± 1,205 for GLM: directly copied files have lower conditional execution-token means than private or initial files.These conditional means do not establish an effect of assigning copied skills; file quality, task difficulty, and selection history may differ.Source: §E.4 · p. 39, Figure 53 · p. 39
  • Observation and copying diagnostics use distinct denominators and conditional summaries, so their rates and execution costs describe the policies’ realized behavior rather than a single resource measure.Observation frequency is divided by available agent-rounds; execution-token means are conditional on acquisition origin and learning period.Source: Figure 54 · p. 39, Figure 55 · p. 40, Figure 56 · p. 40

F WHO OBSERVES WHOM, AND HOW MUCH COMPUTATION DO THOSE DECISIONS USE?

LLM agents concentrate attention on a restricted set of peers, but the observed patterns do not show that familiar relationships make social decisions cheaper. Peer scores and token costs offer limited evidence for explaining the concentration.

F WHO OBSERVES WHOM, AND HOW MUCH COMPUTATION DO THOSE DECISIONS USE?

  • The first eligible peer's score differs from the other eligible peers' mean by −0.11 ± 0.20 points for GPT-OSS and −0.47 ± 0.92 for GLM, not explaining attention concentration.These comparisons do not isolate a causal position bias; shuffled peer orders would be needed to separate position from identity.Source: F WHO OBSERVES WHOM, AND HOW MUCH… · p. 40
  • Within-choice score spreads average 0.19 ± 0.04 points for GPT-OSS UCB and 0.30 ± 0.02 for GLM UCB, versus 0.87 ± 0.11 and 2.51 ± 0.12 in LLM-social populations.The narrower UCB spreads are consistent with copying spreading a small set of highly ranked files, but do not establish why accuracy is similar.Source: F WHO OBSERVES WHOM, AND HOW MUCH… · p. 40
  • Attention favors first-listed peers early and late, yet GPT-OSS usually changes sources within restricted relationships; lifetime link breadth need not mean broad attention within each block.The fixed peer order prevents attributing these preferences solely to skill quality.Source: F WHO OBSERVES WHOM, AND HOW MUCH… · pp. 40–41, Figure 57 · p. 41
  • GPT-OSS's observation-selection calls rise from 254 ± 8 completion tokens early to 387 ± 4 later, while GLM shows no clear temporal pattern.These are decision costs before observation and deployment choice costs afterward, not just the cost of reading a copied file.Source: F WHO OBSERVES WHOM, AND HOW MUCH… · p. 41, Figure 57 · p. 41
  • Directly copied skills are associated with fewer execution tokens early across all three social controllers, but the difference narrows or changes later and is not causal.Agents choose which skills to acquire and use.Source: F WHO OBSERVES WHOM, AND HOW MUCH… · p. 41
  • Familiar/new-source contrasts match within observer and five-round block, but leave candidate quality, prompt contents, and reasons for observing uncontrolled.Only blocks containing both source types contribute, and later observations of new sources may be rare.Source: F WHO OBSERVES WHOM, AND HOW MUCH… · p. 43, Figure 61 · p. 43

G RANDOMIZED TESTS OF COPYING TIME

Randomized timing tests find that forcing observations early does not reproduce GLM's endogenous social advantage, and limiting access to an early window does not improve performance. The reporting safeguards distinguish these schedule comparisons from broader causal or equivalence claims.

G RANDOMIZED TESTS OF COPYING TIME

  • The assigned-timing test fixes each agent at four observations, while the early-access test leaves observation optional and compares with reused unrestricted-social and solo populations.Both use six matched population seeds and the same static IFBench protocol; the early-access contrast can change both timing and copying frequency.Source: G RANDOMIZED TESTS OF COPYING TIME · p. 44, Figure 62 · p. 44
  • For GLM, forced early observations lower final whole-answer accuracy by 2.42 ± 0.46 percentage points versus distributed timing, with average accuracy 0.88 ± 0.50 points lower.Distributed observation finishes 3.00 ± 1.13 points above matched solo learning, versus 0.58 ± 0.95 points for forced early observation.Source: G RANDOMIZED TESTS OF COPYING TIME · p. 44, Figure 63 · p. 45
  • Early-only access lowers average trajectory accuracy versus unrestricted access by 0.54 ± 0.25 points for GPT-OSS and 0.79 ± 0.47 for GLM, without improving final accuracy.Observation counts fall from 17.60 ± 0.69 to 5.70 ± 0.46 for GPT-OSS and from 4.10 ± 0.44 to 2.23 ± 0.10 for GLM; displaced observations are largely replaced by private revisions.Source: G RANDOMIZED TESTS OF COPYING TIME · p. 45, Figure 63 · p. 45, Figure 64 · p. 46
  • Non-initial copied versions make up 79.17 ± 4.73% of forced-early observations for both models, versus 93.33 ± 1.67% for GPT-OSS and 95.83 ± 1.54% under distributed timing.The schedule also changes subsequent search and the skills available, so this does not isolate copying the same artifact at different times or establish an optimal schedule for other tasks or models.Source: G RANDOMIZED TESTS OF COPYING TIME · p. 45, Figure 64 · p. 46

H INTERPRETATION AND REPORTING SAFEGUARDS

  • Timing contrasts are nominal and unadjusted unless stated otherwise, and nonsignificance does not establish equivalence.Table 4's ten finite-option comparisons use Bonferroni correction; it reports cumulative reward through round 100 and within-seed efficiency changes relative to solo.Source: H INTERPRETATION AND REPORTING… · pp. 45, 47, Table 4 · p. 47
  • The uniform source control changes both self-selection frequency and peer identity, so it tests adaptive source allocation as a package rather than peer identity alone.Copied-option comparisons also condition on recipients having a personal alternative.Source: H INTERPRETATION AND REPORTING… · p. 47
  • Figure 65 plots realized finite-bandit reward against completion tokens by population seed; it is not an estimate of the causal return to assigning more tokens.The skill-evolution logs separate several decision, revision, evaluation, execution, and audit costs, but do not isolate the cost of interpreting a copied file.Source: H INTERPRETATION AND REPORTING… · p. 47, Figure 65 · p. 48
Loading 2609.38516v1…