LabBook: Harnessing Experimental History for Efficient LLM-Driven Discovery

• The paper frames the mapping from experimental history to the next proposal context as an important determinant of discovery quality and cost.Source: §1 · pp. 1–2
• Selected-parent evolutionary harnesses keep proposal contexts manageable but can discard useful failures, partial successes, and implementation details when attempts are not chosen as ancestors.Source: §1 · p. 1
• Full-history contexts retain evidence but grow with every attempt and repeatedly include large programs, similar failures, and obsolete hypotheses.Source: §1 · p. 1
• LabBook separates complete experimental logging from selective retrieval, using an agent-maintained memory to guide focused context construction for the next solution.The agent jointly produces the next program and rewritten memory without an explicit population or branching search structure.Source: §1 · p. 2
• Across 49 Frontier-CS problems and 9 additional mathematical and heuristic-design tasks, LabBook achieves comparable or better solution quality than evaluated baselines at lower measured cost.The evaluation also analyzes the quality–cost trade-off across 58 discovery problems and how accumulated evidence guides later proposals.Source: §1 · p. 2

2.1 EVOLUTIONARY AND SELF-IMPROVING SCIENTIFIC DISCOVERY

• Evolutionary discovery systems select archives, parents, or lineages for proposals; later methods refine selection, novelty, quality-diversity, resource allocation, or search adaptation.Examples include FunSearch, AlphaEvolve (Novikov et al., 2025), and ShinkaEvolve, CodeEvolve, AdaEvolve, and EvoX (Lange et al., 2026; Assumpc¸˜ao et al., 2026; Cemri et al., 2026; Liu et al., 2026a).Source: §2.1 · pp. 2–3
• Some systems improve discovery processes or agents themselves, whereas LabBook leaves the generator and its parameters fixed and manages the within-run information path.The cited alternatives include agent-code evolution, semantic deltas, and replay-based exploration-policy improvement.Source: §2.1 · p. 3

2.2 MEMORY AND EXPERIMENTAL-HISTORY REUSE

• Memory research reuses interaction history through natural-language memory, reflection, trajectory insights, hierarchical context management, or appended prior solutions.Examples include Generative Agents (Park et al., 2023), Reflexion (Shinn et al., 2023), ExpeL (Zhao et al., 2024), MemGPT (Packer et al., 2024), and OPRO (Yang et al., 2024).Source: §2.2 · p. 3
• Unlike memory-aware discovery systems with population management or separate agent roles, LabBook uses one agent, one rewritten state, and read-only operations over a lossless log.The same agent selects evidence and jointly produces the next program and updated LabBook.Source: §2.2 · p. 3

3.1 DISCOVERY AS A HISTORY-TO-PROPOSAL MAPPING

• The history-to-context mapping is a distinct design decision: LabBook maintains compact state, retrieves exact evidence from complete history, and supplies context for the next proposal without an explicit population or branching structure.The agent converts that context into a new program.Source: §3.1 · pp. 3–4

3.2 LABBOOK

• LabBook is a single free-form natural-language state recording each approach's idea, decisive implementation details and parameters, lessons learned, and useful unresolved directions.Source: §3.2 · p. 4
• The initial context always includes the latest complete evaluated program and feedback, LabBook's persistent account of earlier experiments, and deterministic progress signals.The progress signals represent effort since the current best was first attained and expose regressions and plateaus; minimization tasks replace score maximization with minimization.Source: §3.2 · p. 4
• The agent can request a score overview, a complete earlier program, or its evaluator feedback through a coarse-to-fine read-only interface, without embeddings or a learned retriever.It may stop when the context is sufficient and is limited to at most K retrieval queries.Source: §3.2 · p. 4

3.3 HARNESS DESIGN

• Simple retrieval fits the setting because the task, program interface, and evaluator remain fixed, making the log a homogeneous, chronologically indexed collection of experiments.LabBook provides a high-level map of explored approaches, lessons, and promising directions.Source: §3.2 · p. 4
• Each iteration constructs the initial context, allows bounded retrieval, evaluates exactly one new program, appends its record to the local history, and rewrites LabBook.The retrieval budget is an upper bound, not a required number of calls.Source: §3.3 · p. 5

4.1 EXPERIMENTAL SETUP / 4.2 EVALUATION METRICS

• Evaluation covers 49 Frontier-CS problems and nine cross-domain tasks, compares OpenEvolve, AdaEvolve, and EvoX on two backbones, and measures benchmark-native quality against API cost.Frontier-CS scores are normalized partial-credit scores; the other tasks use their released task-specific objectives.Source: §4.1 · p. 5, §4.2 · p. 6

4.3 MAIN RESULTS

• At final displayed budgets, LabBook leads the cost–performance trade-off and improves over the strongest baseline by 6.13 points (16.3%) with DeepSeek and 6.91 points (20.9%) with Qwen.It surpasses AdaEvolve’s final score at 30.2% lower cost with DeepSeek and 48.8% lower cost with Qwen.Source: Figure 2 · p. 6, §4.3 · p. 6
• With DeepSeek, LabBook has the highest score and lowest cost on both mathematical tasks; with Qwen, it remains within 1.7% and 0.2% of the best scores, respectively.The two tasks are Circle Packing and Heilbronn Convex.Source: §4.3 · p. 6
• On nine cross-domain tasks, LabBook is best or tied-best on all nine with DeepSeek and seven with Qwen.Table 1 reports each task’s native score and measured cost at 50 iterations.Source: §4.3 · p. 6, Table 1 · p. 7
• On the three ADRS systems tasks, LabBook reaches the best or tied-best score with both backbones and attains PRISM’s proved optimum of 26.2560 with both.With DeepSeek, it also costs less than AdaEvolve and OpenEvolve on EPLB and Transaction Scheduling, and less than AdaEvolve on PRISM.Source: §4.3 · p. 7
• LabBook obtains the best or tied-best result on all four HeuriGym tasks with both backbones.DeepSeek has the lowest cost on E-Graph Extraction and Pedigree; Qwen has the lowest unnormalized objective on E-Graph Extraction, Operator Scheduling, and Pedigree.Source: §4.3 · pp. 7–8

4.4 ABLATION STUDIES

• Across nine-task normalized aggregation, Full LabBook scores highest; Full History costs more than twice as much without improving the aggregate result.On Heilbronn Convex, Full LabBook rises from 0.8942 at iteration 5 to 0.9452 at iteration 30, while No Retrieval plateaus near 0.633.Source: Table 2 · p. 8, §4.4 · p. 8, Figure 3 · p. 8

4.5 CASE STUDY: LONG-HORIZON RECOVERY ON HEILBRONN CONVEX

• At iteration 77, LabBook retrieved iteration 33’s reliable backbone 44 iterations later, added targeted optimization changes, and reached a new best score of 0.947325.The iteration-33 backbone combined softmin continuation, a low-dimensional polar construction, exact-objective Nelder–Mead refinement, pattern search, and simulated annealing.Source: Figure 4 · p. 9, §4.5 · p. 9

5 CONCLUSION

• LabBook separates a complete, append-only experimental log from an agent-maintained state that summarizes what is known and guides selective retrieval.The conclusion presents this as a simple mapping from experimental history to the context for the next proposal.Source: §5 · p. 9
• Across 58 problems, the resulting harness provides a competitive cost–quality trade-off across two language-model backbones.Source: §5 · p. 9
• The conclusion suggests effective discovery need not depend on increasingly elaborate search structures and identifies noisy, partially observed, and non-programmatic workflows as future directions.Source: §5 · p. 9

A APPENDIX

• The appendix also covers token usage and cost accounting, experimental-log and LabBook growth, median Frontier-CS results, prompts, retrieval protocol, and a representative trajectory.Source: A APPENDIX · p. 11
• Table 3 summarizes the 58 evaluated open-ended problems: two mathematical, three systems, four heuristic-design, and 49 Frontier-CS problems.Source: Table 3 · p. 12
• Table 4 lists the selected Frontier-CS problems grouped using the category definitions adopted by EvoX.Source: Table 4 · p. 12

A.1 BENCHMARK COMPOSITION

• The evaluation includes 58 open-ended problems; nine from AlphaEvolve, ADRS, and HeuriGym are listed individually, while the Frontier-CS sample has 10 optimization, 18 constructive, and 21 interactive problems.Source: §A.1 · p. 12
• The Frontier-CS sample includes 10 optimization, 18 constructive, and 21 interactive problems, covering all three problem types adopted by EvoX.Source: §A.1 · p. 12
• Table 4 lists the selected Frontier-CS problem identifiers by category.Source: §A.1 · p. 12

A.2 TOKEN USAGE AND PRICING

• Table 5 reports mean per-problem Frontier-CS token usage at the selected checkpoints, with rounded token counts and costs in USD.Source: §A.2 · p. 12, Table 5 · p. 13
• Table 6 reports growth of the complete experimental log and compact persistent memory over 50 iterations, including their size ratio at iteration 50.Source: Table 6 · p. 13
• DeepSeek comparisons use a $0.35 per-problem ceiling with checkpoints at iterations 30, 60, and 50 for LabBook, AdaEvolve, and EvoX/OpenEvolve; Qwen uses a $1.10 ceiling with iteration 70 for LabBook and 100 for baselines.Comparisons use backbone-specific thresholds because providers have different rate cards and cache policies; they are made within, not across, backbones.Source: §A.2 · p. 13
• LabBook processes more total tokens, but 66.3% are low-priced cache hits; Qwen reports zero cache-read tokens, so its input is charged at the full rate.The passage attributes Qwen’s higher measured cost primarily to the provider not exposing cache-read discounts.Source: §A.2 · p. 13

A.3 EXPERIMENTAL-LOG GROWTH AND MEMORY COMPRESSION

• At iteration 50, LabBook remains 1.38–4.89 kB while logs reach 460.8–1450.5 kB, a log-to-memory ratio of 111–594×.The Frontier-CS row averages across 49 problems; other rows use the corresponding DeepSeek run.Source: §A.3 · p. 13
• The experimental log grows continually as complete records are appended, whereas LabBook does not grow in proportion to the history.Source: §A.3 · p. 13
• Payload size is measured in UTF-8 at iterations 10, 30, and 50.Source: §A.3 · p. 13

A.4 MEDIAN FRONTIER-CS RESULTS

• LabBook obtains the highest median best-so-far Frontier-CS score with both backbones at the final checkpoint shown for each method.Source: §A.4 · p. 14
• The median statistic is computed across the 49 Frontier-CS problems within each run, then averaged across runs.Source: §A.4 · p. 14
• The median results agree with the mean cost–performance curves, indicating that the mean result is not driven only by a few unusually large improvements.Source: §A.4 · p. 14

A.5 PROMPT TEMPLATES

• The prompt has three functional blocks: shared discovery policy, read-only retrieval protocol, and joint program–memory output contract.Runtime-supplied values appear in angle brackets; the task statement and latest iteration context are omitted from the template.Source: §A.5 · pp. 14–15
• The shared policy establishes the best reference, diagnoses the latest attempt using evaluator feedback and prior evidence, then chooses a refinement or a different approach.It also directs the agent to retain lessons from earlier experiments when redirecting.Source: §A.5 · p. 14

A.6 EVOLUTION OF LABBOOK

• At iteration 5, a three-fold symmetric engine crashed because its bounds had length 9 for a 10-parameter representation; the memory classified this as an implementation bug, not evidence against the idea.The bounds were corrected, and the symmetric engine then ran correctly.Source: §A.6 · p. 16
• The best result that RAN achieved was iteration 33 with a score of 0.945; it was identified as the decisive construction, with a plateau lasting 16+ attempts at 0.945.Source: §A.6 · p. 16
• This attempt used softmin L-BFGS-B optimization with bounds [0,1] and temperatures 0.02..0.0005, alongside polar and partial-symmetry engines, simulated annealing, and exact-objective Nelder–Mead polishing.The polar engine used settings (4,3,40,999), (2,6,30,777), and (6,2,20,555); partial-symmetry settings were (4,3,40,2024) and (3,4,30,2025). Annealing used 40,000 iterations with seed 4242; polishing used three rounds, then two.Source: §A.6 · p. 16
• Adding global engines beyond iteration 33’s set scored 0.857 because the extra engines starved the backbone before it reached 0.945.The later proposal kept the iteration-33 backbone’s budgets and added one guarded, budget-neutral SLSQP polish stage.Source: §A.6 · p. 16
• The snapshots show LabBook retaining the iteration-33 anchor, the time-budget lesson, and unresolved directions while dropping most low-level chronology.Its organization and wording are repeatedly rewritten rather than formed by simply appending each latest experiment.Source: §A.6 · p. 17
• By iteration 20, LabBook had consolidated unsuccessful experiments into negative lessons while retaining the strongest construction and its decisive settings.The memory records iteration 18’s best score as 0.909548 and says to keep that construction faithfully.Source: §A.6 · pp. 16–17