MOOSE-Chem3: Toward Experiment-Guided Hypothesis Ranking via Simulated Experimental Feedback

1 INTRODUCTION

• Experiment-guided ranking updates which hypotheses to test next using outcomes from previously tested hypotheses, unlike pre-experiment ranking based on language-model reasoning alone.Iterative real experiments in natural sciences are costly, time-consuming, and throughput-limited, restricting scalable feedback.Source: §1 · p. 2
• The simulator approximates feedback in a local neighborhood of hypothesis space, reflecting the premise that nearby hypotheses can provide a path toward a ground-truth hypothesis.Its foundations assume a dominant local optimum, relate structural or functional similarity to similar outcomes, and allow imperfect similarity representations to distort the performance landscape.Source: §1 · p. 2
• Validated against 124 hypotheses with reported experimental outcomes, the simulator shows high trend alignment and predictive accuracy, with deviations resembling wet-lab noise.It models performance as a function of distance to a hidden ground-truth hypothesis, which remains hidden from the ranking policy.Source: §1 · p. 2
• The clustering-based ICRL policy significantly outperforms pre-experiment baselines and strong ablations.It decomposes hypotheses into functional components, groups them by mechanistic roles, and uses trial feedback to prioritize untested candidates.Source: §1 · p. 3
• The paper frames experiment-guided ranking as a sequential decision problem and introduces an ICRL framework with a clustering-based agentic policy.The policy analyzes tested hypotheses for useful functional components, then prioritizes untested candidates that include promising elements.Source: §1 · p. 3

2.1.1 CONCEPTUAL FOUNDATIONS

• A1 treats the labeled ground truth as a dominant local optimum, P1 links structural or functional similarity to similar outcomes, and D1 predicts distortions from imperfect similarity representations.The resulting landscape may contain noise, spurious optima, or unexpected valleys; the labeled hypothesis is not guaranteed to be the strict local optimum.Source: §2.1.1 · p. 3

2.1.2 MATHEMATICAL FORMULATION

• The ideal model uses a Gaussian-like performance surface that decays with distance from the ground truth, while imperfect embeddings add a correction term that can produce a noisier, multimodal landscape.The correction term represents discrepancies between oracle and practical embeddings.Source: §2.1.2 · pp. 4–5

2.2 A PRACTICAL IMPLEMENTATION OF ϕ(·) WITH PRIOR KNOWLEDGE

• The implementation decomposes the ground-truth and candidate hypotheses into aligned functional components and mechanisms, then assigns ground-truth component weights and labels critical components.The ground-truth decomposition is performed first and guides the candidate decomposition.Source: §2.2 · p. 5
• Component-level mechanism similarities are combined with a weighted sum and a penalty for missing critical components to produce a bounded similarity score used as simulated performance.A perfect match to the ground truth receives a score of 1.Source: §2.2 · pp. 5–6

3.1 TASK FORMULATION

• An effective experiment-guided strategy incorporates prior evaluation feedback and minimizes the expected number of trials needed to identify the optimal hypothesis.The goal reflects the cost and time associated with real or simulated experiments.Source: §3.1 · p. 6

3.2 METHODOLOGY

• CSX-Rank uses a frozen LLM in a gradient-free, non-parametric learning process that relies on forward passes rather than costly fine-tuning.The framework is formulated as sequential decision-making over states, hypothesis selections, and experimental rewards.Source: §3.2 · p. 7
• The agent extracts and classifies functional components, prunes ineffective ones, and clusters the remainder by functional similarity to form a traceable state representation.Each component remains linked to its originating hypothesis.Source: §3.2 · p. 7
• For each selection, the policy first chooses a promising mechanistic cluster using prior domain knowledge, then selects a hypothesis within that cluster to test.The executor returns a normalized score that is analyzed and incorporated into a running summary for the next decision cycle.Source: §3.2 · p. 7
• The component-driven process makes the agent’s decision trail explicit and auditable, from component extraction and clustering through final hypothesis selection.The authors describe this transparency as allowing domain experts to examine the agent’s logic and derive insights.Source: §3.2 · p. 7

4.1 SIMULATOR: EVALUATING THE SIMULATOR WITH REAL EXPERIMENT RESULTS

• CSX-Sim outperforms baselines across all reported simulator metrics, including trend alignment, consistency, and predictive error.The evaluation compares simulated outcomes with experimentally reported results collected from literature.Source: §4.1 · p. 8, Table 1 · p. 8

4.2 EXPERIMENT-GUIDED RANKING: BASELINES AND ABLATION STUDY

• Over 32 trials on average for Uninformed Search versus under 30 for Pre-Experiment Ranking; both remain behind CSX-Rank, which improves sample efficiency.The comparison reports the number of trials needed to identify the ground-truth hypothesis.Source: §4.2 · pp. 8–9, Table 2 · p. 9
• Progressively removing clustering, feedback analysis, and full feedback from CSX-Rank leads to marked performance degradation.The ablations assess the contribution of these components to efficient hypothesis ranking.Source: §4.2 · p. 9, Table 2 · p. 9

4.3 SIMULATOR: ABLATION ON DIFFERENT ϕ(·) WITH DIFFERENT LEVELS OF DISTORTION

• Under Complex Noise, CSX-Rank requires 32.7 trials versus 36.5 without clustering and 40.5 without clustering and analysis; higher noise complexity degrades performance.The evaluation compares CSX-Rank and its ablated variants across noise conditions.Source: §4.3 · p. 9, Table 3 · p. 9

5 RELATED WORK

• Score-based and pairwise pre-experiment ranking methods rely on LLM internal reasoning without incorporating experimental outcomes.The cited examples include Yang et al. (2024; 2025), Zhou et al. (2024), Si et al. (2024), and Liu et al. (2025).Source: §5 · p. 9
• Feedback-loop methods in mathematics and programming refine hypotheses using verification outcomes, while hypothesis-driven feedback elsewhere is often limited to domains with efficient verifiers.The cited examples are Romera-Paredes et al. (2024), Shojaee et al. (2024), Ma et al. (2024), Novikov et al. (2025), and Qiu et al. (2024).Source: §5 · pp. 9–10
• For natural sciences, costly experiments motivate deliberate experiment-guided ranking that maximizes information from each test rather than exhaustive trial and error.The work also focuses on general-purpose simulators for settings where real experiments are costly and feedback is scarce.Source: §5 · p. 10

6 CONCLUSION

• The paper formalizes experiment-guided ranking and proposes a simulator grounded in three domain-informed conceptual foundations to address limited experimental feedback.The simulator is validated against 124 hypotheses.Source: §6 · p. 10
• The clustering-based agentic policy in the ICRL framework significantly outperforms pre-experiment baselines.The framework builds on the simulator and the experiment-guided ranking task.Source: §6 · p. 10
• The simulator and policy together provide a toolkit for advancing feedback-driven hypothesis discovery, with potential impact across the natural sciences.Source: §6 · p. 10

ETHICS STATEMENT

• The authors report following the ICLR Code of Ethics, acknowledge dual-use and dataset or foundation biases, and commit to responsible application.They state that no human subjects were involved.Source: ETHICS STATEMENT · p. 10

A.1 A FRAMEWORK FOR EXTRACTING CRITICAL FUNCTIONAL COMPONENTS IN THE SIMULATOR

• For chemistry problems, the extraction framework identifies key components, analyzes their roles and mechanisms, then reports components and conclusions.Figure 5 is identified as the framework for extracting chemical components.Source: §A.1 · p. 12, Figure 5 · p. 12

A.2 PROMPT FOR EXTRACTING KEY CHEMICAL COMPONENTS IN THE SIMULATOR

• The prompt distinguishes key components from prerequisites and validation methods, requests mechanistic explanations, and separates extracted components from experimental results.It also asks the model to retain specific concentrations or mass fractions and group substances that function together.Source: §A.2 · pp. 12–13
• In the thermocell example, CSX-Sim identifies five of the human expert’s six key points, missing PVA as the hydrogel matrix.The simulator’s extracted points align with those proposed by the expert, according to the comparison.Source: §A.2 · p. 15

B THE ROLE OF CRITICALPOINTS IN CSX-Sim

• The CriticalPoints example assigns R = 0 when a hypothesis omits the redox pair considered essential to the ground-truth thermoelectric function.The corresponding wet-lab results are described as nonzero for the first three hypotheses and zero for the fourth.Source: B THE ROLE OF CRITICALPOINTS IN CSX-Sim · p. 16

C EVALUATING THE SIMULATOR WITH REAL EXPERIMENT RESULTS

• The simulator validation compares simulated outcomes with real experimental outcomes using a dataset of 124 chemical hypotheses.The section describes this as an evaluation of predictive performance and reliability.Source: C EVALUATING THE SIMULATOR WITH REAL… · p. 16

C.1 DATASET COMPOSITION AND ANALYSIS

• The evaluation dataset comprises 124 authentic wet-lab experiment results: 16 from Polymer Chemistry, 36 Organic, 33 Inorganic, and 39 Analytical Chemistry, spanning four chemistry subfields.Source: §C.1 · p. 17

C.2 TREND COMPARISON WITH REAL EXPERIMENT RESULTS

• Figure 6 orders 124 experiments by experimental result and compares normalized outcomes; each simulated result averages three CSX-Sim trials.The vertical axis shows normalized experimental results.Source: §C.2 · p. 17, Figure 6 · p. 18
• Across all 30 question sets, CSX-Sim predicts mean trends consistent with real experimental means (bootstrap p < 0.01).Figure 6 compares simulated and real trends; its error bars represent population standard deviation.Source: §C.2 · pp. 17–18, Figure 6 · p. 18
• Normalization removes scale differences from the comparison, supporting assessment of trend matching across the 124 results.The passage describes close correspondence between simulated and real mean data.Source: §C.2 · p. 18

D DISCIPLINES OF THE TOMATO-CHEM BENCHMARK DATASET

• Table 6 is identified as the disciplinary classification of the TOMATO-chem benchmark dataset.The supplied passage gives the table’s scope but no category counts.Source: Table 6 · p. 18

E.1 EVALUATION OF TREND ALIGNMENT

• CSX-Sim achieves a mean Spearman correlation of ρ = 0.960, indicating close alignment between simulated and experimental rankings.Spearman correlation measures the monotonic relationship between simulated and experimental outcome rankings.Source: §E.1 · p. 19
• CSX-Sim reaches perfect trend alignment (ρ = 1) in 26 of 30 question groups, as measured by PCI.PCI counts groups with an exact match in simulated and experimental rankings.Source: §E.1 · p. 19

E.2 EVALUATION OF SIMULATOR ACCURACY

• CSX-Sim beats Matched Score across metrics (MSE 0.058, MAE 0.161, RMSLE 0.147); removing ComponentExtraction degrades results more than removing CriticalPoints.Without CriticalPoints, MSE is 0.064, MAE 0.174, and RMSLE 0.159; without ComponentExtraction, the values are 0.087, 0.215, and 0.192, respectively.Source: Table 7 · p. 19, §E.2 · p. 20

F DIFFERENT LEVELS OF DISTORTION

• The simulator groups discrepancies into Simple, Moderate, and Complex Noise, with the top 35%, middle 40%, and bottom 25% assigned to those levels.The levels model increasingly challenging feedback conditions.Source: F DIFFERENT LEVELS OF DISTORTION · pp. 20–21
• As noise complexity increases, experiment-guided ranking requires more simulation-feedback iterations to identify the ground-truth hypothesis.The passage attributes this to greater discrepancy and erroneous information in complex-noise feedback.Source: F DIFFERENT LEVELS OF DISTORTION · p. 21

G PERFORMANCE COMPARISON OF DIFFERENT FUNCTIONS IN THE SIMULATOR

• All tested monotonic function forms provide effective ranking guidance, supporting the framework’s adaptability to different function choices.The comparison evaluates Spearman correlation, RMSE, and Perfect Consistency.Source: Table 9 · p. 21, G PERFORMANCE COMPARISON OF DIFFERENT… · p. 21

• Experiment-guided ranking markedly improves over pre-experiment ranking, using prior feedback to select hypotheses more precisely and mitigate accumulating biases.The method outperforms pre-experiment baselines and strong ablations.Source: H EVALUATION OF EXPERIMENT-GUIDED… · p. 22
• The simulator models performance from ground-truth similarity plus noise, while pseudo-ranking clusters hypotheses by shared functional characteristics and uses simulated feedback to prioritize candidates.It was evaluated against 124 chemistry hypotheses with experimentally reported outcomes.Source: H EVALUATION OF EXPERIMENT-GUIDED… · p. 22
• The approach may reduce extensive wet-lab testing costs and accelerate development of materials and drugs, with possible benefits for healthcare access and environmental sustainability.Source: H EVALUATION OF EXPERIMENT-GUIDED… · p. 22

I SCALABILITY ANALYSIS

• At 128 candidates, CSX-Sim required 30.7 trials versus 64.5 for Uninformed Search and 51.3 for Pre-experiment ranking.Source: Table 10 · p. 22, I SCALABILITY ANALYSIS · p. 22
• CSX-Sim required 15.2 trials for 64 candidates; CSX-Rank showed near-linear O(N) growth, with an observed slope of approximately 0.24 × N.The authors report that these results align with their theoretical analysis.Source: I SCALABILITY ANALYSIS · pp. 22–23
• When clustering is highly effective, the theoretical best-case cost could fall to O(log N) because one experiment’s evidence can generalize across its cluster.Source: I SCALABILITY ANALYSIS · p. 23