Source-linked AI summary
SkillGLoW: Procedural-Family Skill Consolidation for Self-Improving Agents on Long-Horizon Task Streams
Ao Yan, Xin Zhang, Jiawei Du, Joey Tianyi Zhou
TL;DR
Evidence on self-generated skills has concentrated on structurally similar tasks, leaving open how to organize skills for heterogeneous long-horizon workloads. SkillGLoW groups local skills into procedural families, compresses shared procedures into global priors, and regenerates task-specific details. Across 12 continual-improvement runs, frozen priors gained 17.2 points (hard) over No-Skill on average, improving in every run.
Problem
Prior work on self-generated skills has focused mainly on structurally similar tasks, leaving open how agents should organize skills for long-horizon streams of heterogeneous tasks.
Source: §1 · p. 1Method
SkillGLoW groups execution-derived local skills into procedural families, compresses shared procedures into frozen global priors, regenerates task-specific detail, and commits updates only after execution-based verification.
Source: §1 · p. 2Results
Across 12 continual-improvement runs, frozen priors gained 17.2 points (hard) over No-Skill on average, improving in every run.
Source: §4.2 · p. 5Takeaways & Limitations
Unmodified priors raised success on unseen embodied tasks from 73.9% to 83.9%, supporting transfer of procedure rather than task memory.
Source: §1 · p. 2A large procedural family had no net gain on its eight tasks, indicating that outcomes can depend on how families are drawn.
Source: §4.4 · p. 7
Abstract
from arXiv · showhide
LLM agents increasingly self-improve by writing and reusing textual skills, kept either as one global document or as a flat pool of per-task entries, though most of the evidence comes from domains with structurally similar tasks. On long-horizon workloads where each task demands a different solution, the two forms fail in opposite ways: the document collapses into generic discipline, while the pool inflates and its entries stay bound to the instance that wrote them. We argue the missing unit of reuse is the solving procedure shared by a cluster of related tasks, and build SkillGLoW (Global-Local Weave) around it: the local skills a task writes from its own execution are aggregated into procedural families and compressed into de-instantiated global priors, while the instance detail they hold is regenerated per task rather than stored; a commit gate admits a prior only when real execution shows it does not degrade the deployed library. Across four benchmarks (mathematical reasoning, terminal automation, software repair, and embodied control) and three models, the priors gain 17.2 points (hard) over the no-skill baseline on average, with positive gains in all 12 continual-improvement runs, and 18.0 with local regeneration, while the library holds one prior per procedural family, 3.6x more compact than the per-task pool. Under the same protocol GLoW leads a published single-document optimizer on 15 of 21 cells. Unmodified, the library lifts success on unseen ALFWorld tasks from 73.9% to 83.9%, evidence that what transfers is procedure rather than task memory.
1 Introduction
On heterogeneous long-horizon tasks, neither a single global skill document nor a flat per-task library captures what transfers: related tasks share procedures, while instance details vary. SkillGLoW consolidates shared procedures into global priors and regenerates local details for each task, committing updates only when real execution shows no degradation.
- On SkillsBench, human-curated skills raise pass rate by 16.2 points, while task-description-only model-written skills land 1.3 points below the no-skill baseline (Li et al. 2026a).Experience-driven methods ground skills in execution trajectories, but prior evidence concentrates on structurally similar tasks, where little beyond meta-level experience transfers (Kim et al. 2026).Source: §1 · p. 1
- Single-document and per-task organizations fail on heterogeneous workloads because tasks lack one dominant procedure and few stored entries fit new tasks, while related tasks still share a solving procedure.The paper reports that these alternatives, including a pool with retrieval, each gain less than family consolidation on the same streams.Source: §1 · p. 2
- GLoW groups execution evidence into procedural families, compresses each shared procedure into a stable global prior, and regenerates task-specific local detail from execution feedback.The prior stays frozen during solving; offline revisions are committed only when real execution shows they do not degrade the deployed library.Source: §1 · p. 2
- 17.2 points: GLoW’s committed priors improve over the no-skill baseline on average, with positive gains in all 12 continual-improvement runs.Across three models and four benchmarks, the deployed library holds one prior per procedural family and is 3.6× more compact than the per-task pool.Source: §1 · p. 2
- 10.0 percentage points: on unseen embodied tasks, frozen priors raise mean success from 73.9% to 83.9%.The paper presents this as evidence that the consolidated content is reusable procedure rather than task memory.Source: §1 · p. 2
2 Related Work
Prior work focuses either on producing textual skills or on organizing and maintaining skill libraries. This work shifts the question to what granularity of skill should be maintained across families of tasks.
Skill construction and optimization
- Textual skill methods mine or pool trajectories, or optimize a held-out-validated document; this work instead asks what granularity of generated skills should be maintained across task families.Earlier systems discover executable skills through exploration (Wang et al. 2023; Zheng et al. 2025); textual systems include AutoSkill (Yang et al. 2026b), Trace2Skill (Ni et al. 2026), ExpeL (Zhao et al. 2024), and SkillOpt (Yang et al. 2026a), which optimizes one document for a single domain.Source: §2 · p. 2
Skill organization, transfer, and abstraction
- Existing skill and procedural-memory systems store units at varied granularities, but evidence largely assumes similar tasks, where cross-domain retrieval can transfer negatively and abstract memories travel better.XSkill pairs action-level experiences with task-level skills (Jiang et al. 2026); unlike XSkill's retrieval and adaptation of past records, this work derives local skills from current-task feedback and never writes them back.Source: §2 · p. 2
Lifecycle and multi-skill maintenance
- Lifecycle systems govern library edits using utility scorers; GLoW instead re-derives its global layer each round from local evidence and accepts it based on measured downstream execution.SkillOS frames self-evolution as library governance (Ouyang et al. 2026a), while SkillGraph adds prerequisite structure to retrieval (Li et al. 2026b).Source: §2 · p. 2
3 Method
GLoW turns execution-grounded local skills into de-instantiated priors for procedural families, admitting library revisions only when verifier scores support them. At execution, it combines a frozen recalled prior with a freshly generated local skill.
3.1 Problem Formulation
- GLoW treats the verifier score as the trustworthy measure of a skill’s effect on a frozen agent and maintains one global prior per procedural family rather than per historical task.The library size therefore tracks distinct procedures, not the number of tasks seen.Source: §3.1 · pp. 2–3
3.2 Local Evidence and Skill Cards
- Localize derives a task-local skill from changes between real executions and verifier feedback, then records the task instruction, solution signature, trajectory, verifier score, and skill in a card.Local skills use only the current task’s execution feedback and are not written directly into the long-term library.Source: §3.2 · p. 3, §3.1 · p. 3
3.3 Procedural Clustering and Prior Compression
- GLoW clusters local skills by solving procedure, using consensus co-occurrence across clusterings and automatic Kneedle selection to form families, then compresses each family into a de-instantiated prior.Priors retain applicability conditions, the core procedure, and common failure modes; task-specific details are regenerated in local skills.Source: §3.3 · pp. 3–4
3.4 Verifier-Grounded Commit
- GLoW commits candidate revisions only when verifier-grounded execution shows they meet the standing library and no-skill anchor, allowing ϵ = 0.02 for measurement fluctuation.Candidates come from fresh compression, append-only repair of degraded priors, and the best-scoring prior so far; the gate makes one decision for the whole revision per round.Source: §3.4 · p. 4
3.5 Execution-Time Global–Local Weaving
- At test time, Recall injects the base prior and the top-1 similar family prior only when similarity exceeds a fixed threshold; the frozen context is combined with a freshly generated, updated local skill.Recall finishes before execution and remains fixed throughout the task; local skills are staged as evidence for later consolidation and admission.Source: §3.5 · p. 4
4 Experiments
Across 12 continual-improvement runs, frozen procedural-family priors improve on No-Skill, with further gains from local regeneration; the paper also reports transfer to unseen tasks and a smaller committed library than the per-task pool.
4.1 Setup
- Experiments use four benchmarks spanning 32, 20, 42, and 53 tasks, with three frozen base models across 12 continual-improvement runs.Settings isolate No-Skill, local-only adaptation, global-only priors, and Global+Local; retrieval uses a cosine-similarity threshold of 0.45.Source: §4.1 · pp. 4–5
4.2 Main Results
- Frozen priors gain 17.2/13.0 points (hard/soft) over No-Skill on average, rising to 18.0/14.6 with local regeneration across 12 runs.Both GLoW settings improve on No-Skill in all 12 runs (Wilcoxon signed-rank p = 0.000488).Source: §4.2 · p. 5, Table 1 · p. 5
- Across 21 cells, GLoW leads SkillOpt on 15, SkillOpt leads on 4, and two tie; SkillOpt’s four wins are all on ALFWorld.On TBP, SWE-bench, and LMB, GLoW’s hard-score margins average +8.3, +6.7, and +6.9 points, respectively.Source: §4.2 · pp. 5–6
4.3 Component Ablations
- Base-only averages +2.0 points and the per-task pool +10.9, while one round of family consolidation gives +11.2 points.Base-only ranges from −9.3 to +16.7 and loses ground on 4 of 12 cells; AWM averages +5.0 across 12 cells, with eight up, three down, and one flat.Source: §4.3 · p. 6, Table 2 · p. 6
- The gate accepts 19 and rejects 7 of 26 candidates, and its final libraries average +14.7 points versus +9.6 for that round’s candidates.Four of the seven rejected candidates would have passed a consolidation-time-score-only criterion.Source: §4.3 · p. 6, Figure 3 · p. 6
4.4 Analysis
- ALFWorld’s 42 tasks yield 15 priors covering all six dataset classes, while Terminal-Bench-Pro’s 32 tasks yield ten procedure classes.The byte-scanning prior helps only the two tasks actually solved by scanning raw bytes, leaving three other binary-work tasks unchanged.Source: §4.4 · pp. 6–7, Table 3 · p. 7
- An examine-in-light prior compresses a 989-word, instance-specific local skill into 440 words of four-step procedure, while one mixed eight-task family has no net gain.The prior replaces memorized furniture locations with checking the visible object list; families are re-derived each round, so a bad grouping is not carried forward.Source: §4.4 · p. 7
4.5 Transfer to Unseen Tasks
- All three models improve on 60 unseen ALFWorld tasks, while MiniMax-M3’s software-repair resolve rate rises from 40.0% to 45.6% on 30 unseen instances.The ALFWorld split shares task categories with training but no instances; the software-repair result averages three trials.Source: §4.5 · p. 7, Table 4 · p. 7
5 Conclusion
SkillGLoW consolidates local skills into de-instantiated procedural-family priors, regenerates instance details per task, and admits priors only when measured execution supports them.
5 Conclusion
- 17.2 points over No-Skill, a library 3.6× smaller than the per-task pool, and unseen-task transfer rises from 73.9% to 83.9%.Transfer was shown where task categories recur.Source: §5 · p. 7
- The approach compresses local skills from each task cluster into one de-instantiated prior, regenerates instance detail per task, and admits priors through measured execution.Source: §5 · p. 7
- Whether priors survive a genuine domain change remains open, and cross-model inheritance of plain-text libraries remains untested.The conclusion frames both as open questions for libraries intended to support open-ended workflows.Source: §5 · p. 7
A Metric Definitions … C.1 SkillOpt
The appendix defines the benchmark-specific hard and soft scores and explains why soft scores guide commits while hard scores headline results. It also documents procedural-skill clustering and the controls used to compare SkillOpt with the authors’ method.
A Metric Definitions
- The hard metric is the benchmark’s all-or-nothing flag; the soft metric is its continuous partial credit, except ALFWorld’s verifier-derived soft score.Table A1 presents both metrics one benchmark at a time.Source: A Metric Definitions · p. 10, Table A1 · p. 10
- SWE soft scoring is multiplicative, while Avg. soft averages different scales and supports only within-row relative comparisons; per-benchmark soft columns are comparable and hard columns are unaffected.A SWE score is zero if either side of its multiplicative gate is zero, so breaking a previously passing test is not partial success.Source: A Metric Definitions · p. 10, Table A1 · p. 10
- With No-Skill hard scores from 13.2% to 76.2%, the commit gate uses soft scores, while headline results report hard scores.The cited candidate and deployed scores, 0.283 and 0.151, are both soft.Source: A Metric Definitions · p. 10
B Multi-View Fusion and Consensus Clustering
- Each local skill card contributes up to five L2-normalized view vectors to a weighted sum; the body describes four views, with two signature views encoded at different granularities.The runtime tier depends on which fields the round’s cards contain, and the twelve runs use the two tiers in Table A3.Source: B Multi-View Fusion and Consensus… · p. 10, Table A2 · p. 10, Table A3 · p. 10
- The deployed tiers assign 0.50 total weight to signature channels and the least weight to full skill text; missing channels fall back to task instructions, and vectors use an EMA with α = 0.2.Tier weights are hard-coded across four benchmarks and three models; the four-view weights come from a separate sweep scored by ARI, whereas the five-view tier is the optimum of a five-view grid sweep.Source: B Multi-View Fusion and Consensus… · p. 10, Table A3 · p. 10
C.1 SkillOpt
- The SkillOpt comparison uses its official implementation and defaults, while Table A5 aligns the task set, base model, trials, thinking, turn limit, and scoring.The reported defaults are learning rate 4, minibatch 8, four epochs, and batch 40; skill-length budget and prompts are untouched.Source: §C.1 · p. 11, Table A5 · p. 11
- On MiniMax-M3 × ALFWorld, blank and hand-written starts scored identically across three epochs (0.0 pp), while the optimization loop produced +19.0 pp.All four benchmarks use the same neutral 99-byte skeleton, and SkillOpt’s optimizer is set to the solver’s base model, removing distillation on both sides.Source: §C.1 · p. 11
- All four benchmarks use the evaluation adapter because SkillOpt’s native ALFWorld and LMB read-outs are not comparable with the paper’s; SkillOpt’s train peak merges epoch batches using weights.Native ALFWorld sets soft to 1 on success, and native LMB scores 0.077 on the same 53 tasks where the No-Skill baseline scores 0.226; through the adapter, SkillOpt scores higher than in its native environment.Source: §C.1 · p. 11
C.2 AWM · D Statistical Tests
AWM induces shared workflows from prior task experience and deploys them as a global document, but its average gain is smaller than GLoW’s Global arm on the same twelve cells. The comparison highlights the value of family consolidation, execution-based admission, and per-task recall.
C.2 AWM
- AWM uses an offline batch of already-attempted tasks to extract recurring workflows in one LLM call, then prepends the result whole to the agent prompt.Its induction instruction is used verbatim, while the example is replaced with one using the target benchmark’s action vocabulary; induction has no web-specific assumption.Source: §C.2 · p. 11
- AWM induces from No-Skill rollouts—the same blank-skill experience available to every arm—before any SkillOpt update.These are the first-step rollouts of a SkillOpt run started from an empty skill document, matching the No-Skill condition in Table 1.Source: §C.2 · pp. 11–12
- AWM pools global retrieval results and injects the same top-k document into every task, whereas GLoW’s Recall is per task.AWM sets top-k to 3 rather than the official default of 10, uses text-embedding-ada-002, and deploys the workflow document as the initial skill for one epoch over the task stream.Source: §C.2 · p. 12
- AWM averages +5.0 across twelve cells (range −6.2 to +20.0), while GLoW’s Global arm averages +17.2 and is same-signed in 12/12 cells.The +5.0 comparison is the round-0 committed library’s real deployment score in the first sub-round of round 1, measured against the same cell’s No-Skill result on hard tasks.Source: Table A6 · p. 12, §C.2 · p. 12
- On LMB, AWM’s three model gains are +1.9, 0.0, and −1.9 (mean zero), versus GLoW Global’s +24.5, +5.7, and +3.8 in the same cells.The paper attributes this contrast to AWM inducing over the task set as a whole without first forming procedural families; LMB has the fewest mergeable tasks.Source: §C.2 · p. 12
- AWM workflows enter reuse without an admission step grounded in real execution, unlike GLoW’s commit gate.The margins of candidates rejected in §D.3 range from −0.05 to −0.13.Source: §C.2 · p. 12
D.1 Paired Tests Across Runs … F Held-Out Transfer Results
Paired-run results show Global+Local positive across all 12 runs, while the peak read-out counts only committed, deployed libraries. The gate requires real deployment evidence, and the final global library is reported as 3.6× compressed relative to the local-skill pool.
D.1 Paired Tests Across Runs
- Global+Local was positive in all 12 runs, with a mean of +18.0 pp; the paired tests give p = 2/2^12, the minimum attainable at n = 12.Significance is established at the level of the twelve runs; per-cell resolution is reported in §J.2.Source: §D.1 · p. 12
D.2 What the Peak Read-out Is Taken Over
- In the last round, the deployed trajectory was +14.7 pp versus +9.6 pp for the candidate, leaving the retained library 5.1 pp above the new candidate.GLoW's peak is taken only across libraries that passed the commit gate and were deployed; rejected candidates do not enter that pool.Source: §D.2 · p. 12
D.3 Gate Decisions and the Tolerance
- Held-out evaluation does not enter gate decisions; each candidate is adjudicated after the first real deployment of the next round, and every reported prior has deployment evidence.The first-round anchor reduces to the no-skill baseline because no committed round exists yet.Source: §D.3 · p. 13
- With ϵ = 0.02, two TBP candidates at −1.08 and −1.81 pp were admitted within tolerance, below the 2.71 pp reproduction SD; rejected candidates were −5 to −13 pp.Equality passes by design, but whether admitting a tie helps or hurts downstream was not tested.Source: §D.3 · p. 13
E Library-Size Accounting
- Compression compares last-round committed global priors with the same-round final local-skill pool; entry ratios count entries, while word ratios compare summed word counts.Intermediate local-skill snapshots are excluded; including them would raise the ALFWorld denominator from 42 entries to 168.Source: E Library-Size Accounting · p. 13
- All four read-out combinations yielded ratios of 0.28–0.31; the body reports 3.6× compression, versus 3.8× pooled-word and 4.1× mean per-cell reciprocal aggregations.The 3.6× figure is the least favorable of the three reported aggregations.Source: Table A9 · p. 13, Table A10 · p. 13, E Library-Size Accounting · p. 13
F.1 ALFWorld Held-Out · F.2 SWE Held-Out · G What the Committed Priors Contain
On held-out tasks, the committed prior is evaluated without modification: ALFWorld uses the frozen global prior alone, while SWE reports a 40.0→45.6 resolve-rate increase. The ALFWorld results move in the same direction across all three models, with MiniMax-M3 gaining least despite starting highest.
F.1 ALFWorld Held-Out
- ALFWorld tests the committed global prior alone, without local regeneration, on the official 60-task unseen split at a single trial.Table A11 identifies the held-out results as hard (%).Source: §F.1 · p. 13, Table A11 · p. 13
- All three ALFWorld models move in the same direction, while MiniMax-M3 starts highest and gains least, consistent with a ceiling effect.The supplied passages do not provide the individual model values.Source: §F.1 · p. 13
- The ALFWorld library was not modified for unseen tasks; the injected prior is the version committed on the training stream.Source: §F.1 · p. 13
F.2 SWE Held-Out
- 40.0→45.6: MiniMax-M3’s mean resolve rate increased on SWE’s 30-task held-out split, a +5.6 pp change.The change comes from matched three-trial runs; the split is disjoint from train20.Source: §F.2 · p. 13, Table A12
- The SWE held-out comparison uses MiniMax-M3 and mean resolve rate over three trials.The 30-task held-out split is disjoint from train20.Source: §F.2 · p. 13
- The SWE result is based on a matched pair of held-out runs, one without skills and one with the global prior.The supplied run labels identify the pair as no-skill and global round 0.Source: §F.2 · p. 13, Table A12
G.1 Agreement With the Dataset’s Labels · G.2 The Family Mapping for the Terminal Benchmark · G.3 Excerpts of Prior Text
These appendices compare procedural families with ALFWorld’s labels, map families to terminal-task procedures, and show how compression ranges from generic guidance to executable task-solving steps.
G.1 Agreement With the Dataset’s Labels
- Purity remained 0.943 after removing five singleton families, while middling ARI reflected intentional splitting into 15 families against six task types.ALFWorld has seven tasks in each of its six independently labeled types; the passage attributes the ARI gap to over-segmentation, not type mixing.Source: Table A13 · p. 14, §G.1 · p. 14
- Two families cross-cut the goal taxonomy where goals differ but procedures match, including Egg tasks that swap cooling-then-microwaving for heating-then-refrigerating.The passage describes the appliance and receptacle as swapped in this clearer example.Source: §G.1 · p. 14, Table A14 · p. 14
G.2 The Family Mapping for the Terminal Benchmark
- Ten terminal families cover 32 tasks, with nine repairs against one pass-to-fail change in the round-2 mapping.A dash in the mapping means no task moved in that direction.Source: Table A15 · p. 14
- In the stripped-binary family, tasks split by whether their procedure operates at byte level, and the marker-scanning prior repaired only the two byte-level tasks.The prior scans markers, validates candidate manifests, and decrypts bitmaps; the test uses each task’s trajectory-derived solution and verifier-determined pass status.Source: §G.2 · p. 14, Table A16
G.3 Excerpts of Prior Text
- A non-main-table, pre-partitioned 16-task TBP run produced generic debugging guidance, while a finer binary family retained domain-level procedural steps.The debugging prior was 500 words and characterized its two unrelated-domain tasks as sharing only generic engineering discipline; the binary prior was 650 words.Source: §G.3 · p. 15
- The ALFWorld family prior compressed seven local skills into four procedural steps and 440 words, replacing a memorized location table with a single observation read.Its actions include going to the source receptacle and taking the target directly, avoiding a separate examination turn; the merge note says it chose between mutually exclusive strategies rather than concatenating them.Source: §G.3 · p. 15
H Benchmarks and Splits … K Hyperparameters
The section fixes shared task splits and a uniform execution and hyperparameter protocol, then measures reproduction variability to contextualize run-level gains and single-cell readings.
H Benchmarks and Splits
- All four streams use fixed shared split files; seed=42 controls TBP, SWE, and LMB, while the ALFWorld selection is deterministic and seed-independent.The files record split names, seeds, and sizes; LMB and ALFWorld files also include task-identifier hashes.Source: H Benchmarks and Splits · p. 15, Table A17 · p. 15
- Seed=42 controls sampled splits and task traversal, not generation; vendor APIs use default decoding temperature, measured at 0.7.There are exactly two held-out evaluations, from different origins and with different evidential weight.Source: H Benchmarks and Splits · p. 15, Table A18 · p. 15
I Run Artefacts and Execution Volume
- Each of the 12 cells is an independent continual run with three rounds, three sub-rounds, and one trial per task; all use the same loop size.Per-cell supplementary directories include summaries, configurations, library states, task results, committed priors, and local skills.Source: I Run Artefacts and Execution Volume · p. 16, Table A19 · p. 16
- Searching three candidate routes uses no extra rollouts; scores come from nine existing deployments, and admission is decided on the next round’s first deployment.Across 1,323 task-×-round pairs, the second and third sub-rounds improve a task’s best local skill 162 times (12.2%) and 70 times (5.3%), respectively.Source: I Run Artefacts and Execution Volume · p. 16, Table A19 · p. 16
J.1 Reproduction Dispersion of a Single Trial
- 17.2 points is the main table’s mean Global gain over no-skill, 6.4 times the 2.71 pp pooled within-group SD; all 12 runs share a sign.The dispersion estimate comes from 18 round-0 measurements in six groups matching on split, model, trial count, and time limit.Source: §J.1 · p. 16, Table A20 · p. 16
J.2 Per-Cell Resolution
- ±5.4 pp defines single-cell reproduction resolution at two standard deviations, based on 18 real repetitions rather than an i.i.d.-task assumption.An i.i.d. assumption would yield ±9 to 15 pp for 42 or 53 tasks, two to three times wider than the measured scale.Source: §J.2 · p. 16
K Hyperparameters
- One hyperparameter setting is shared across all four benchmarks and three models, with no cell-specific tuning; the paper reports no sensitivity analysis.The complete hyperparameter table is Table A21.Source: K Hyperparameters · p. 16, Table A21
L Compute Environment … N.1 Size and Distribution
The appendix documents the compute constraints, benchmark-independent consolidation prompts, and how procedural-family structure is selected and counted. It also presents family-size distributions for one run per benchmark.
L Compute Environment
- At 24 GB, a fully loaded SWE run exhausts host memory, so the WSL2 ceiling was lowered to 21 GB, bounding SWE and TBP concurrency.The environment is summarized in Table A22.Source: L Compute Environment · p. 17, Table A22 · p. 17
- The GPU is used only for embedding retrieval and does not participate in base-model inference.Source: L Compute Environment · p. 17
M Consolidation-Pipeline Prompts
- The consolidation pipeline uses 24 prompt templates, all phrased without benchmark names or task identifiers.Source: M Consolidation-Pipeline Prompts · p. 17
- Five core prompts are deployed in pipeline order across the twelve runs; none contains a benchmark name, task identifier, or answer.Their single domain interface is a placeholder filled at runtime with the benchmark category phrase; family names are generated during compression rather than drawn from a preset taxonomy.Source: M Consolidation-Pipeline Prompts · p. 17
N Family Structure
- ALFWorld’s selected run is based on ARI against dataset labels, while the other three benchmarks use size-distribution balance × (1 − singleton fraction).Source: N Family Structure · p. 17
- Prior counts include one base prior injected into every library, so they equal the number of families K plus one; Table 3 counts family priors only.Source: N Family Structure · p. 17
N.1 Size and Distribution
- Table A23 presents family structure for one run per benchmark.Source: Table A23 · p. 17
N.2 Family Names and the Clustering Axis … O.2 A Flat Retrieval Library
Family names can obscure the procedural basis of clustering, while append-only repair has model-dependent effects. On LMB validation, the family library matched the flat library’s top score with fewer entries.
N.2 Family Names and the Clustering Axis
- Family names are generated after clustering from salient member wording, so they do not identify the clustering axis; TBP runs even produced different partitions.The cited TBP × MiniMax-M3 run had K = 9 families, including one with eight tasks, while the cited GPT-5.4-mini run had K = 10 with at most five tasks per family.Source: §N.2 · pp. 17–18
- Family members share procedure signatures rather than subject matter: LMB families group proof-ordering or classification procedures, while SWE families group expression-table or downstream-guard procedures.The reported multi-member families cross branches of mathematics in LMB and repositories in SWE.Source: §N.2 · p. 18, Table A24 · p. 18
N.3 Per-Family Gains
- ALFWorld × MiniMax-M3’s family composition includes five singleton families: pick_and_place, two pick_clean families, pick_two, and pick_heat.Source: §N.3 · p. 19, Table A25 · p. 19
O.1 Prior Repair
- Append-only repair changed soft performance by ↑3.1 for DeepSeek, ↓3.4 for GPT, and ↑4.8 for MiniMax-M3, so its effect is not guaranteed to help.All three candidate routes go through the same commit gate, which adjudicates repair through real execution rather than a fixed priority order.Source: §O.1 · p. 20
O Other Ways to Organise a Library
- The flat-library control used LMB’s 15-task validation split and held the model, injection channel, and retriever constant, changing only how the library was built.Source: §O.2 · p. 20
O.2 A Flat Retrieval Library
- 0.467: the family library reached the LMB validation group’s top score with 0.42× the entries, matching a flat library 2.4 times its size.Both family-library arms were above the median of the four flat-library arms; this was a 15-task validation split at a single trial.Source: Table A28 · p. 20, §O.2 · p. 20
Related papers
This paper cites
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?2023
- Self-Refine: Iterative Refinement with Self-Feedback2023
- Voyager: An Open-Ended Embodied Agent with Large Language Models2023
- SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering2024
- Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models2025
- Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces2026