PaperlayerView exact revision on arXiv

Source-linked AI summary

Language Models as AI Research World Models

Zijun Wang, Zewen Liu, Minhua Lin, Zhaotian Weng, Zhan Shi, Bing He, Yisi Sang, Dakuo Wang, Benoit Dumoulin, Wei Jin, Yuyin Zhou, Cihang Xie, Hanqing Lu

arXiv:2610.12235v1cs.CL

TL;DR

Under limited experimental budgets, predicting intervention outcomes matters because proposed experiments can outpace the capacity to execute them. This paper studies language models as Research World Models that predict outcomes across research environments using experimental experience. Research knowledge improves prediction and selection, including multi-round gains under a fixed selection budget.

  • Problem

    AI research agents can propose experiments faster than they can execute them, leaving researchers to anticipate intervention outcomes when allocating limited training or evaluation budgets.

    Source: §1 · p. 1

  • Method

    The paper studies language models as shared Research World Models that predict candidate-intervention outcomes from the target environment and available experimental history.

    Source: §1 · pp. 1–2

  • Results

    Across prediction and selection settings, research knowledge improves RWM performance; under 24 selections, in-env and cross-env knowledge raise mean final best gain by 15.8% and 11.6%, respectively.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    Experimental records yield reusable knowledge that benefits subsequent research within and beyond their source environments, motivating research-data collection and dedicated RWM training.

    Source: §1 · p. 3

    Demonstrated benefits arise through in-context learning, and cross-environment evidence is currently limited to autoregressive pretraining.

    Source: §5 · p. 9

Abstract

from arXiv · show

AI research agents automate the cycle of proposing, implementing, and evaluating experiments, opening a path toward recursive self-improvement. Yet their ability to propose experiments outpaces their capacity to execute them in real environments, making outcome prediction a key capability for sustained self-improvement under limited experimental budgets. We investigate language models as Research World Models (RWMs), which predict the outcomes of candidate interventions across research environments. Our evaluation draws on over 2,600 experimental records from nine research environments spanning pretraining, post-training, and inference, representing more than 171,000 H100 GPU-hours of experimentation. Research knowledge acquired from real experimental experience improves RWM predictions of unseen interventions within the same environment (Spearman +0.27), and can be reused across environments. For example, using only pretraining experience from OLMo3, Marin, and Nanochat, an RWM reduces selection regret in the Qwen3 environment by 78% compared with zero-experience setting. These benefits extend to multi-round Autoresearch under a fixed selection budget: RWMs with in-env and cross-env research knowledge increase the best gain achieved by 15.8% and 11.6%, respectively. Ablations across 13 language models used as RWMs show that adding research knowledge can improve intervention ranking more than changing models or increasing reasoning effort alone. These findings support language models as RWMs and motivate accumulating experimental data for future RWM training.

1 INTRODUCTION

Research World Models use language models to predict candidate-intervention outcomes across research environments, drawing on experimental records to inform decisions. The evaluation finds that research knowledge improves prediction and selection, including in multi-round Autoresearch under a fixed budget.

1 INTRODUCTION

  • Across five in-env environments, research knowledge raises mean Spearman correlation on unseen interventions from 0.506 to 0.774.Across five pretraining environments, cross-env knowledge raises mean Spearman correlation from 0.628 to 0.729 without target-environment records.Source: §1 · p. 2
  • When selecting three of sixteen candidates, in-env knowledge raises mean NDCG@3 from 0.599 to 0.785, while cross-env knowledge reduces macro-average Regret@3 by 52.2%.The regret reduction is across five autoregressive pretraining environments.Source: §1 · p. 2
  • Under the same 24-selection budget, in-env and cross-env research knowledge increase mean final best gain by 15.8% and 11.6%, respectively.Both gains are relative to no initial research knowledge.Source: §1 · p. 3
  • Across 13 language models, adding research knowledge improves intervention ranking more than changing models or increasing reasoning effort alone.With the model fixed, MAX effort raises Spearman correlation from 0.609 to 0.648, while adding research knowledge at LOW effort raises it to 0.892.Source: §1 · p. 3
  • The results support accumulating experimental data because records can benefit subsequent research within and beyond the environment that produced them.The work acquires and reuses this knowledge through in-context learning and offers an empirical basis for collecting research data and training dedicated models.Source: §1 · p. 3

2 LANGUAGE MODELS AS RESEARCH WORLD MODELS

The paper models research tasks as environments with fixed reference systems and conditions, then uses language models to predict intervention gains from task context and experimental history. It evaluates whether records from the same or other environments can inform those predictions.

2.1 RESEARCH ENVIRONMENTS AND INTERVENTIONS

  • Each research environment specifies a reference system and execution and evaluation conditions; interventions are executable changes relative to that reference.The study fixes each environment's reference while history accumulates, and cross-environment transfer tests usefulness when fixed conditions such as the reference model or training implementation change.Source: §2.1 · p. 3

2.2 LLMS AS RESEARCH WORLD MODELS

  • A shared RWM predicts the median gain of an intervention's rollout in a specified environment, conditioning on the available history.Language models implement this role by interpreting task descriptions, interventions, code, configurations, and experimental records; their predictions are assessed for forecast accuracy, transfer, and experiment-selection value.Source: §2.2 · pp. 3–4

2.3 LEARNING FROM AND TRANSFERRING EXPERIMENTAL EXPERIENCE

  • Research history is represented as experiment records that retain task, intervention, terminal-state, and gain information for in-context learning.Evaluations compare no history, same-environment history, and other-environment history; queried outcomes are withheld, and model backbones or reasoning effort are compared using matched queries and histories.Source: §2.3 · p. 4

3 EXPERIMENTS

Across nine research environments and 2,653 experimental records, the experiments test whether research knowledge improves predictions and whether those gains transfer to experiment selection and multi-round research. Knowledge improves multi-round outcomes under the same selection budget, while also strengthening prediction and selection across environments.

3.2 RESEARCH KNOWLEDGE IMPROVES RWM PREDICTIONS

  • The prediction evaluation tests whether research knowledge improves outcome prediction within and across environments before assessing downstream selection and multi-round outcomes.The evaluation spans pretraining, post-training, and inference.Source: §3.1 · p. 4

3.1 EXPERIMENTAL SETUP

  • The evaluation covers nine research environments and 2,653 records representing 171,862 H100 GPU-hours, with a fixed backbone and inference settings across record conditions.The default RWM backbone is Claude Opus 5 with a 1M context window and HIGH reasoning effort; W0 receives no experimental records, while Win-env receives records from the evaluated environment.Source: §3.1 · pp. 4–5, Table 1 · p. 5

3.2.1 IN-ENVIRONMENT PREDICTION (IN-ENV)

  • Within-environment records increased mean Spearman correlation by 0.27 over W0 across five environments and outperformed kNN and ridge on both prediction metrics in every environment.Table 2 presents within-environment prediction performance for fixed splits, with history and held-out candidate counts identified in its caption.Source: §3.2.1 · p. 5, Table 2 · p. 6

3.2.2 CROSS-ENVIRONMENT PREDICTION (CROSS-ENV)

  • Source records improved Spearman point estimates in all five transfer targets, but kNN averaged 0.36 versus W0’s 0.63 across targets.The authors note that similar interventions can have different effects under different target recipes; for a Nanochat Muon variant, kNN predicted +0.018 BPB while the measured gain was −0.036 and W0 predicted −0.035.Source: §3.2.2 · pp. 5–6, Table 3 · p. 6

3.3 RWM PREDICTIONS IMPROVE EXPERIMENT SELECTION

  • Research knowledge reduced within-environment Regret@3 by a mean 58.2% and cross-environment macro-average Regret@3 by 52.2% versus W0.NDCG@3 improved in every within-environment evaluation; both selection metrics improved in every cross-environment target except Marin.Source: §3.3 · p. 6, Table 4 · p. 7, Table 5 · p. 7

3.4 MULTI-ROUND AUTORESEARCH

  • 15.8% and 11.6% higher mean final best gain came from in-environment and cross-environment records, respectively, versus W0 under a 24-selection budget.Cross-environment records approached within-environment performance and outperformed feature-based predictors using the same source records.Source: §3.4 · p. 8, Figure 2 · p. 8

3.5 RESEARCH KNOWLEDGE, BACKBONE CHOICE, AND REASONING EFFORT

  • Experimental records improved rankings across all 13 RWM backbones, and even LOW reasoning effort with records outperformed every effort setting without records.The lowest-performing backbone with records exceeded the highest-performing backbone without them.Source: §3.5 · pp. 8–9, Figure 3 · p. 8

4 RELATED WORK

Related work spans automated research workflows, predictive evaluation of candidates, and world models for planning. This paper positions outcome forecasting across research environments as support for experiment allocation under limited budgets.

4 RELATED WORK

  • Automated research agents streamline proposal, implementation, and evaluation, but candidate experiments still consume time and computational resources to assess.The paper studies prediction as a complementary way to help agents decide which candidates merit execution under limited budgets.Source: §4 · p. 9
  • Prior predictive approaches use historical evaluations, candidate preferences, or pairwise solution comparisons to support optimization and research search.Examples include LLAMBO, research preference models, and Predict-Before-Executing.Source: §4 · p. 9
  • World models predict action consequences for planning, while this work studies whether experimental knowledge improves intervention forecasts within and across research environments.It also evaluates whether improved forecasts translate into better experiment selection under fixed budgets.Source: §4 · p. 9

5 DISCUSSION AND FUTURE WORK

The discussion presents experimental records as reusable knowledge, while noting limits in demonstrated transfer and the costs and risks of applying research prediction. Extended related work connects the study to automated agents, prediction methods, in-context learning, and world models.

5 DISCUSSION AND FUTURE WORK

  • Experimental records can improve prediction and guide subsequent research, including across environments, with demonstrated cross-environment benefits currently limited to autoregressive pretraining.The authors suggest preserving environment descriptions, interventions, and outcomes, including experiments that do not improve performance; the demonstrated benefits arise through in-context learning.Source: §5 · p. 9
  • Training RWMs on diverse experimental records and comparing generalization with in-context learning in unseen environments are proposed as next steps.The authors also call for evaluating research progress and total costs, since better predictions alone do not guarantee more effective research.Source: §5 · p. 9
  • RWM prediction and allocation could accelerate safety and efficiency research or capability development, and could concentrate advantages among organizations with large experimental archives.Constructing records and evaluating RWMs also consumes substantial compute, which the paper reports to contextualize efficiency claims.Source: ETHICS STATEMENT · p. 10

A EXTENDED RELATED WORK

  • Automated research systems span idea generation, experimentation, writing, and multi-step evaluation, while this work focuses on forecasting outcomes to prioritize experiments.The related work includes systems with different search and execution workflows, alongside benchmarks for machine-learning experimentation and engineering.Source: A EXTENDED RELATED WORK · p. 13
  • Predictive approaches range from surrogate models and learning curves to language-model forecasts, but do not resolve prediction for arbitrary code-level interventions.Prior work includes partial-execution predictors, LLAMBO, preference models, and pairwise forecasts; reliability and information needs vary by task.Source: A EXTENDED RELATED WORK · pp. 13–14
  • In-context learning conditions frozen language models on examples, but its mechanisms and controlled evidence do not guarantee transfer across arbitrary research settings.The paper distinguishes predicting outcomes of proposed interventions from learning adaptive action selection from histories.Source: A EXTENDED RELATED WORK · p. 14
  • World-model precedents predict action consequences for planning, while this study assesses measured gains of complete research rollouts rather than multi-step control or observation reconstruction.The paper also distinguishes its action-conditioned outcome prediction from another use of “Research World Model” as a persistent knowledge graph.Source: A EXTENDED RELATED WORK · pp. 14–15

B.1 ENVIRONMENTS AND BENCHMARK SCORING

The environments use distinct reference models, interventions, evaluation protocols, and gain definitions. The appendix also reports reference variability to contextualize small measured gains.

B.1 ENVIRONMENTS AND BENCHMARK SCORING

  • Benchmark scores are absolute measurements, while gains are computed relative to each environment’s own reference.Autoregressive pretraining uses mean BPB across ten tasks, with specified few-shot contexts, and defines gain as reference BPB minus candidate BPB.Source: §B.1 · p. 15
  • OLMo3 environments use the official OLMo-core implementation, while Marin, Qwen3, and Nanochat are adapted study-specific variants trained from random initialization.OLMo3-100M denotes 100M non-embedding parameters; the three scaled variants have approximately 153M, 153M, and 149M total parameters, respectively.Source: §B.1 · p. 15
  • Diffusion pretraining scores greedy full-window DUEL BPB, with gain defined as reference BPB minus candidate BPB.Evaluation uses 800 sequences per rank.Source: §B.1 · p. 15
  • Math distillation uses sampled-solution accuracy, code RL uses the LiveCodeBench macro score, and inference optimization uses guarded decode throughput.Their gains are respectively candidate-minus-reference accuracy in percentage points, candidate-minus-reference LiveCodeBench score in percentage points, and effective throughput minus reference throughput.Source: §B.1 · p. 15
  • Reference-run variability provides context for interpreting small gains but does not estimate intervention-specific variance.The appendix reports repeatability measurements across seeds or independent runs for autoregressive pretraining, math distillation, and inference optimization.Source: §B.1 · p. 15

B.2 EXPERIMENTAL RECORDS

The prediction pipeline conditions language models and baselines on permitted experimental records while withholding candidate outcomes. The appendix specifies record construction, prediction prompts, baseline tuning, and matched backbone and reasoning-effort comparisons.

B.2 EXPERIMENTAL RECORDS

  • Each experimental record represents one execution, with benchmark components retained as readouts and execution status retained when gain is undefined.Table 6 summarizes the retained record fields.Source: §B.2 · p. 16, Table 6 · p. 16

C.1 INPUT CONSTRUCTION AND PREDICTION PROMPTS

  • Queries combine the target environment and reference, a pre-execution intervention description, and permitted records while withholding the queried outcome and post-run metadata.The prompt asks for a conditional median gain estimate and specifies the reference, budget, metric, and aggregation rule.Source: §C.1 · p. 16
  • Transfer prompts group historical evidence by environment and require predictions for the stated target, accounting for differences in reference recipes and mechanism effects.Held-out interventions and their near-duplicate groups are excluded from source histories in the reported cross-environment evaluation.Source: §C.1 · p. 16

C.2 BASELINE PREDICTORS AND HYPERPARAMETER SELECTION

  • The kNN baseline predicts the median gain of nearest neighbors, and baseline hyperparameters are selected by validation MAE using permitted history.Cross-environment validation groups records by semantic intervention group; held-out outcomes are excluded from tuning and fitting.Source: §C.2 · p. 17

C.3 BACKBONE AND REASONING-EFFORT COMPARISONS

  • Backbone and reasoning-effort comparisons use matched queries and histories, with model and inference settings held fixed where specified.The effort sweep compares LOW through MAX with the default Claude Opus 5 context window; zero-record and record-conditioned forecasts use matched inference settings.Source: §C.3 · p. 17

D STATIC EVALUATION AND EXPERIMENT SELECTION

Cross-environment prediction uncertainty is estimated with paired bootstrap intervals over semantic groups, and Table 7 reports cross-environment Spearman differences against comparators.

D.1 UNCERTAINTY IN CROSS-ENVIRONMENT PREDICTIONS

  • 3,000 paired bootstrap resamples draw with replacement from 69 semantic groups containing the 71 test interventions.All interventions in each sampled group are retained, and every predictor uses the same resample.Source: §D.1 · p. 17
  • The 2.5th and 97.5th percentiles of resampled Spearman differences define the confidence limits.These pointwise intervals are conditional on the fixed histories, predictions, and recorded outcomes.Source: §D.1 · p. 17
  • Table 7 reports cross-environment Spearman differences between Wcross−env and each comparator, with pointwise 95% paired bootstrap confidence intervals.The intervals are conditional on the fixed histories, predictions, and recorded outcomes.Source: Table 7 · p. 17, §D.1 · p. 17

D.2 SELECTION METRICS AND AGGREGATION

This section specifies selection metrics and aggregation rules, including how ties, ineligible pools, and cross-environment results are handled.

D.2 SELECTION METRICS AND AGGREGATION

  • Selection metrics take the uniform expectation over exact prediction ties.Random selection reports the exact expected score from uniformly selecting three candidates.Source: §D.2 · p. 18
  • Pools with no positive gain are excluded from NDCG@3 but retained for Regret@3.Regret@3 is defined for candidate pool C and selected interventions S3, but the supplied passage does not include its full formula.Source: §D.2 · pp. 17–18
  • Candidate identifiers are sorted before random partitioning into complete pools of 16, with any remainder unused in that partition.Methods share pools and select three candidates before outcomes are revealed; metrics are averaged over eligible pools within each partition and then over 200 partitions.Source: §D.2 · p. 18
  • Cross-environment results are aggregated with equal target weights, while the two reported relative reductions differ in whether they average per-environment reductions or reduce macro-average Regret@3.Aggregates and improvements are computed before rounding.Source: §D.2 · p. 18

E MULTI-ROUND AUTORESEARCH EVALUATION

The multi-round Autoresearch evaluations use shared candidate arrivals and selection schedules, with forecasts updated using the initial history and outcomes from earlier selections.

E.1 CANDIDATE SAMPLING AND ARRIVALS

  • Five evaluations each sample 128 interventions from 231 unseen OLMo3-100M candidates, with 16 new candidates arriving per round across eight rounds.Candidates are organized by mechanism complexity; unselected candidates remain available, and ties are resolved by ascending fixed candidate ID.Source: §E.1 · p. 19
  • All methods share each evaluation's candidate set and arrival schedule, and select from that set before outcomes are revealed.The sampled pool is disjoint from the initial within-environment history.Source: §E.1 · p. 19

E.2 EXPERIMENTAL FEEDBACK

  • Each round's forecast is conditioned on the initial experimental records and all records accumulated through that method's earlier selections.The selection budget starts after the initial experimental records are available.Source: §E.2 · p. 19

E.3 LEARNING-CURVE METRICS

Autoresearch learning-curve metrics track the best gain achieved through selected interventions and summarize performance over the evaluation rounds.

E.3 LEARNING-CURVE METRICS

  • The best-so-far curve tracks the highest gain among interventions selected through each round.Gain is measured relative to the reference.Source: §E.3 · p. 19
  • The unchanged reference is retained as a zero-gain option, and final best gain is the curve value at T = 8 rounds.Arrival Oracle selects the available interventions with the highest measured gains under the same arrival schedule and budget, providing an upper bound for the curve.Source: §E.3 · p. 19
  • Trapezoidal curve area is summarized by averaging separately normalized ratios over five Autoresearch evaluations.Source: §E.3 · p. 19

Related papers

Loading 2610.12235v1…