Source-linked AI summary
Can Test-Time Memory Evolution Outperform Test-Time Training for Scientific Discovery?
Yepeng Liu, Shengmai Chen, Xuandong Zhao, Chengzhi Liu, Dawn Song, Xin Eric Wang, Yuheng Bu
TL;DR
Online weight updates in test-time training are computationally costly, motivating more efficient adaptation for scientific discovery. The paper introduces TTME, which distills search experience into evolving external memory for a fixed-weight model, and reports better scores than TTT across the evaluated tasks with lower runtime and FLOPs.
Problem
Test-time training can outperform inference-only systems on scientific discovery, but its costly online weight updates motivate a more efficient adaptation method.
Source: §1 · p. 2Method
TTME continually distills evaluated search experience into external memory and uses relevant content to guide subsequent discovery without updating model weights.
Source: §1 · p. 2Results
Across two frameworks, three models, and five mathematical tasks, TTME achieves better scores than TTT in every evaluated setting, reducing runtime by 47.7–58.4% and FLOPs by 82.8–84.6% under TTT-Discover with Qwen3-8B.
Source: §5 · p. 10Takeaways & Limitations
The results establish self-evolving memory as an effective and computationally efficient alternative to test-time weight updates for the discovery tasks studied.
Source: §5 · p. 10The study focuses on tasks with executable candidate programs and a task-specific verifier that assesses construction validity and objective value.
Source: §3.1 · p. 3
Abstract
from arXiv · showhide
Recent \emph{test-time training} (TTT) frameworks have enabled open-weight large language models (LLMs) to outperform inference-only frameworks powered by frontier models on scientific discovery tasks. However, TTT introduces additional computational cost for online weight updates during the discovery process, making computational efficiency an important consideration when scaling to larger models. This motivates us to explore an efficient \emph{test-time adaptation} method for scientific discovery. We introduce \emph{Test-Time Memory Evolution} (TTME), a plug-and-play test-time adaptation method, as an alternative to TTT. TTME continually updates the external memory by distilling the experience to guide subsequent discovery without updating model weights. To have a clean comparison, we evaluate TTME and TTT under the same discovery framework, replacing only the test-time adaptation method. Our evaluation shows that TTME outperforms TTT on all five mathematical tasks, while reducing computational cost by $82.8$-$84.6\%$ in FLOPs and $47.7$-$58.4\%$ in runtime. These results establish TTME as an effective and computationally efficient alternative to TTT in the evaluated discovery settings.
1 Introduction
Test-time training uses online weight updates to learn from evaluated search attempts, but its computational cost motivates a more efficient adaptation method. TTME evolves external memory instead, and the paper reports better results than TTT across the evaluated mathematical tasks with lower costs.
1 Introduction
- TTT learns from evaluated attempts through reinforcement learning that internalizes search experience in model weights, but its online updates are costly and time-consuming.The paper identifies this cost as a scaling challenge for larger models and more complex discovery tasks.Source: §1 · p. 2
- TTME continually distills evaluation feedback into external memory and supplies relevant content to guide subsequent discovery with a fixed-weight model.The approach is presented as a plug-and-play alternative to online weight updates.Source: §1 · p. 2
- Guideline Memory captures operational lessons from execution and evaluation, while Strategy Memory extracts score-improving algorithmic ideas for reuse in later exploration.Relevant strategies can combine ideas from different search attempts.Source: §1 · p. 2
- TTME outperforms TTT on all five evaluated tasks, reducing running time by 47.7–58.4% and costs by 82.8–84.6% relative to TTT.The evaluation replaces only TTT within the discovery frameworks.Source: §1 · p. 2
- Both TTME and TTT improve implementation validity over the underlying framework, while TTME performs better and more often builds on score-improving combinations of prior ideas.The paper analyzes the methods’ search behavior during discovery.Source: §1 · p. 2
2 Related Work
Prior non-parametric discovery work organizes search and uses feedback to improve exploration, while parametric methods use reinforcement learning to adapt model weights. This paper positions TTME as a modular memory-based adaptation mechanism and reports better outcomes and lower costs than online weight adaptation.
Non-Parametric Adaptation
- Non-parametric methods use fixed-weight LLMs and evaluation feedback to guide search; prior work adjusts exploration, allocates computation, or uses execution feedback to select modifications.The paper studies experience accumulation as a modular adaptation mechanism through TTME.Source: §2 · pp. 2–3
Parametric Adaptation
3 Method
The method frames scientific discovery as iterative executable-program search, then replaces test-time weight updates with evolving external memories while preserving each discovery framework. Guidelines capture implementation lessons, strategies retain score-improving ideas, and strategy-guided generation adapts those ideas to selected parents.
3.1 Background and Problem Formulation
- TTME retains each framework’s archive and parent-selection rules while replacing test-time RL weight updates with external memory that conditions generation and keeps model weights fixed.The search evaluates candidate programs and records validity, scores, and execution feedback for subsequent updates.Source: §3.1 · pp. 3–4
3.2 Guideline Memory: Shared Operational Lessons
- Guideline Memory distills execution and evaluation feedback into shared implementation lessons, updating entries by recurrence and recency while removing stale items and enforcing a capacity limit.Updates use outcomes collected across parents at intervals of K search steps.Source: §3.2 · p. 4
3.3 Strategy Memory: Reusable Algorithmic Experience
- Strategy Memory extracts ideas from valid modifications that score higher than their parents, then consolidates related experiences into reusable strategy families.Entries retain application conditions and different implementations, with references to their source cases.Source: §3.3 · p. 5
3.4 Strategy-Guided Search and Candidate Generation
- For each parent, a selector chooses up to N relevant strategies and an adaptor turns each into a parent-specific instruction; generation also reserves αB candidates for direct exploration.The experiments set α = 1/2, splitting the per-parent candidate budget between direct and strategy-guided generation.Source: §3.4 · pp. 5–6
4 Experiments
Across two discovery frameworks, three models, and five tasks, TTME outperforms TTT and search without TTT while reducing runtime and computation. Ablations and search analyses report contributions from both memory components and more frequent selection of productive idea combinations.
4.1 Experiment Settings
- The experiments compare fixed-weight search, TTT weight updates, and TTME memory updates across five mathematical construction tasks and three model configurations.For TTME, the model weights remain fixed while the external memories guide subsequent generation.Source: §4.1 · pp. 6–7
4.2 Main Results
- TTME outperforms TTT and search without TTT across two frameworks, three models, and five tasks.The comparisons are reported under TTT-Discover and ThetaEvolve (Tables 1 and 2).Source: §4.2 · p. 7, Table 1 · p. 7, Table 2 · p. 7
- TTME surpassed final TTT scores within 25 steps on all three TTT-Discover tasks and both models, and on three ThetaEvolve tasks.Figure 3's orange dots mark the first step at which TTME exceeds the corresponding final TTT score.Source: §4.2 · p. 7, Figure 3 · p. 8
- 47.7–58.4% runtime and 82.8–84.6% FLOPs are saved by TTME relative to TTT across three TTT-Discover tasks with Qwen3-8B.Runtime excludes evaluation; the computation accounting includes candidate generation, memory-related LLM calls, and RL training where applicable.Source: §4.2 · p. 8, Table 3 · p. 8
4.3 Ablation Studies
- TTME alone achieves the best scores on AC2 and Erdős; adding TTT worsens both results.The comparison covers neither method, TTT alone, TTME alone, and their combination.Source: §4.3 · p. 8, Table 4 · p. 9
- Full TTME achieves the best scores on all three tasks, and removing either Guideline Memory or Strategy Memory worsens performance.The ablation is under TTT-Discover with gpt-oss-120b.Source: §4.3 · pp. 8–9, Table 5 · p. 9
4.4 Search Behavior Analysis
- Both TTT and TTME improve mean validity over search without TTT; on AC1 and Erdős, validity is similar but TTME's best-found scores are higher.A candidate is valid if it compiles successfully and receives an evaluator score; Figure 4 summarizes mean validity over 50 steps and best-found scores.Source: §4.4 · p. 9, Figure 4 · p. 9
- TTME selects combinations more often: 10.2% versus 8.2% on AC1 and 24.7% versus 18.6% on AC2; for combinations improving >10^-5, shares are 5.4% versus 4.8% and 17.1% versus 13.3%.A combination retains a parent's idea and adds one explored outside its ancestry; these combinations were more often selected for further search.Source: §4.4 · pp. 9–10, Figure 5 · p. 10
5 Conclusion
The paper asks whether evolving external memory can outperform test-time training for scientific discovery within the same frameworks. TTME scores better than TTT across the evaluated settings and, in one setting, reduces runtime and FLOPs.
5 Conclusion
- TTME achieves better scores than TTT across two frameworks, three models, and all five evaluated mathematical tasks.The evaluation compares methods within the same discovery frameworks.Source: §5 · p. 10
- Under TTT-Discover with Qwen3-8B, TTME reduces runtime excluding evaluation by 47.7–58.4% and FLOPs by 82.8–84.6%.The FLOPs figure includes all memory-related calls.Source: §5 · p. 10
- The results establish self-evolving memory as an effective and computationally efficient alternative to test-time weight updates for the studied discovery tasks.Source: §5 · p. 10
A Task Definitions and Evaluation
This appendix defines five mathematical construction tasks, including three autocorrelation inequalities and the Erdős minimum overlap problem. The tasks specify objectives over functions or densities, with the autocorrelation passages describing discrete or continuous formulations.
A Task Definitions and Evaluation
- The evaluated tasks include three autocorrelation inequalities, Erdős’ minimum overlap problem, and the maximal determinant problem, with scores retaining each task’s original objective direction.Section 3.1 instead uses a higher-is-better convention for s(x).Source: A Task Definitions and Evaluation · p. 14
A.1 First Autocorrelation Inequality (AC1)
- AC1 minimizes a ratio, with its score defined for heights on equal-width intervals and the autoconvolution denoting discrete convolution.Source: §A.1 · p. 14
A.2 Second Autocorrelation Inequality (AC2)
- AC2 maximizes a ratio for the same class of step functions and computes the squared L2 norm by integrating over each linear segment of the autoconvolution.Source: §A.2 · p. 14
A.3 Third Autocorrelation Inequality (AC3)
- AC3 optimizes over real-valued step functions whose heights may be positive or negative, and its objective is to minimize a quantity.Source: §A.3 · p. 14
A.4 Erdős’ Minimum Overlap Problem
- The Erdős formulation uses a step function and its complement, extended by zero outside the interval, to minimize the maximum overlap.Source: §A.4 · p. 15
A.5 Maximal Determinant Problem (Hadamard)
The maximal determinant task searches for a matrix with the largest absolute determinant and maximizes a normalized score. Evaluation uses exact integer arithmetic for the determinant and tracks best-found scores among valid candidates.
A.5 Maximal Determinant Problem (Hadamard)
- The maximal determinant problem searches for a matrix with the largest absolute determinant.Source: §A.5 · p. 15
- The task maximizes the normalized score used in ThetaEvolve [6].Source: §A.5 · p. 15
- The evaluator computes the determinant using exact integer arithmetic.Source: §A.5 · p. 15
- Best-found scores are computed over all valid candidates evaluated up to the current search step.Source: Table 6 · p. 15
B Experimental Setup
The experimental setup compares TTME and TTT while preserving key framework settings. Each run maintains its own search archive.
B Experimental Setup
- Within each framework, the TTME and no-TTT conditions use the same fixed-weight model, task initialization, evaluator, and primary candidate budget.Source: B Experimental Setup · p. 16
- Each run maintains its own search archive.Source: B Experimental Setup · p. 16
B.1 ThetaEvolve Framework
The ThetaEvolve experiments use frozen Distill-Qwen3-8B on AC2, AC3, and the n = 29 Hadamard problem. They compare TTME settings with ThetaEvolve’s released inference-only configuration.
B.1 ThetaEvolve Framework
- The experiments use frozen Distill-Qwen3-8B on AC2, AC3, and the n = 29 Hadamard maximal-determinant problem.Source: §B.1 · p. 16
- Each step selects 32 MAP-Elites parents and generates 16 primary candidates per parent, with TTME dividing candidates between two generation branches.Prompts use released expert context, the parent program, and SEARCH/REPLACE rules; evaluation results update the archive.Source: §B.1 · p. 16
- Table 7 compares settings against ThetaEvolve’s 65-step early inference-only configuration, with counts reported per task and seed.Memory maintenance, strategy selection, and adaptation calls add to the 33,280 primary candidate attempts.Source: Table 7 · p. 16, §B.1 · p. 16
B.2 TTT-Discover Framework
The TTT-Discover experiments evaluate Qwen3-8B and gpt-oss-120b on AC1, AC2, and Erdős. Both models use the specified search and evaluation settings, and Table 8 compares gpt-oss-120b settings with the original configuration.
B.2 TTT-Discover Framework
- Qwen3-8B and gpt-oss-120b are evaluated on AC1, AC2, and Erdős using TTT-Discover.Table 8 compares gpt-oss-120b settings with the original TTT-Discover configuration; counts are per run.Source: §B.2 · p. 17, Table 8 · p. 17
- PUCT selects eight parents per step, each generating 64 candidates, with TTME allocating 32 candidates to each generation branch.The experiments retain released task prompts, construction-state reuse, and verifier; the buffer allows up to two children per parent and retains initial states.Source: §B.2 · p. 17
- Runs use a 1,000-second program budget and 1,100-second evaluator timeout, with failed programs receiving no retry using error feedback.Each run has 25,600 primary candidate attempts, and TTME adds memory-update, strategy-selection, and adaptation calls.Source: §B.2 · p. 17
C Memory Implementation Details
TTME maintains complementary guideline and strategy memories, updating them from evaluated search experience to guide later candidate implementation and score improvement. The implementation describes their update procedures and the prompts for extracting, consolidating, selecting, and adapting reusable strategies.
C.1 Memory Overview
- TTME separates search experience into shared implementation guidelines and reusable algorithmic strategies that are selected and adapted for subsequent search.Both memories start empty and are updated together every K = 2 search steps; full TTME enables both components.Source: C Memory Implementation Details · p. 18, §C.1 · p. 18, Table 9 · p. 18
C.2 Memory Updates
- Guideline updates summarize recent valid results, scores, and failures, then propose up to three trigger–mechanism–action lessons, with at most one for failure avoidance.Guidelines are matched at Jaccard similarity ≥ 0.5, expire after more than 20 steps without reaffirmation, and are capped at 24 entries ranked by recurrence and recency.Source: §C.2 · p. 18
- Guidelines are shared with both generation modes, which display up to 240 characters per guideline while memory retains the full lesson.The count and text-length limits are configurable hyperparameters.Source: §C.2 · p. 18, §C.3 · p. 19, Table 10 · p. 19
- Strategy notes come from verifier-passing child programs that improve on finite-scored parents, with the fixed-weight model describing the algorithmic change and its application conditions.Notes retain source-case references; independent changes can produce separate notes, and related ideas are grouped into families with implementation variants preserved.Source: §C.2 · p. 19
D.3 Strategy Memory: Strategy Entry Consolidation
- Strategy-family consolidation records the mechanism, distinct execution modes, when each mode suits a program, and what may be adapted, without inventing unsupported thresholds.Entries combine new notes with prior family descriptions while retaining distinct modes; the examples contrast L-BFGS-B refinements placed around up-sampling and Adam polishing.Source: §C.2 · p. 19, §D.3 · p. 22
D.2 Strategy Memory: Experience Note Extraction
- The experience-note prompt asks what changed, where it acts, and which change plausibly connects to the score difference, then requests a concise reusable memo.Averaging gradients across tied maximum-convolution entries is an example of a strategy note; independent bundled changes receive separate answer blocks.Source: §D.2 · p. 21
D.4 Strategy-Guided Search: Strategy Selection
- The selector chooses up to N different strategies for each parent using program context, score history, prior lineage attempts, and methods already assigned to other groups.The caller requests the smaller of N and the available strategy families; selection favors distinct proposed changes rather than duplicate methods.Source: §D.4 · p. 23
D.5 Strategy-Guided Search: Strategy Adaptation
- For a selected strategy, the adaptation prompt commits to one mode suited to the program’s structure and produces a parent-specific modification direction.The input templates supply program context, execution summaries when available, scores, lineage history, strategy memory, and recent outcomes to the relevant memory operations.Source: §D.5 · p. 24, §D.6 · pp. 25–26
E Discovered Constructions
Figure 6 presents discovered constructions for the five tasks defined in Appendix A. The Hadamard panel reports a specific 29 × 29 sign matrix and its objective value.
E Discovered Constructions
- Figure 6 shows discovered constructions for the five tasks defined in Appendix A, including AC1–AC3, Erdős, and Hadamard.The AC1 inset shows normalized heights from 0 to 0.06, and arrows indicate improvement direction.Source: E Discovered Constructions · p. 27, Figure 6 · pp. 27–28
- 0.579938205: Distill-Qwen3-8B with memory enabled produced a 29 × 29 Hadamard sign matrix with this objective value.Dark and light cells encode +1 and −1, respectively.Source: E Discovered Constructions · p. 28
- Panel (e) preserves the original row and column order of the Hadamard construction.Source: Figure 6 · p. 28