PaperlayerView exact revision on arXiv

Source-linked AI summary

MOOSE-Star: Unlocking Tractable Training for Scientific Discovery by Breaking the Complexity Barrier

Zonglin Yang, Lidong Bing

arXiv:2603.03756v4cs.LGcs.CEcs.CL

TL;DR

Scientific-discovery LLM research has largely emphasized inference or feedback-driven training, leaving direct modeling of P(h|b) unresolved because inspiration retrieval and composition create combinatorial complexity. MOOSE-Star decomposes discovery into trainable subtasks and uses hierarchical search and bounded composition, achieving scalable training and continuous test-time scaling while brute-force sampling reaches a complexity wall.

  • Problem

    Existing work rarely trains LLMs to directly model P(h|b), and direct end-to-end modeling is ill-posed because the search space over k inspirations scales as O(N^k).

  • Method

    MOOSE-Star trains decomposed retrieval and composition subtasks, uses hierarchical search for logarithmic best-case retrieval, and applies bounded composition to tolerate retrieval noise.

  • Results

    MOOSE-Star scales continuously with training data and inference budget, whereas brute-force sampling hits a complexity wall for hypotheses requiring multiple inspirations.

  • Takeaways & Limitations

    The framework provides a tractable approach to training P(h|b) while supporting more scalable inference for scientific discovery.

  • Takeaways & Limitations

    The motivation-planning complexity reduction assumes that the generated motivation is sound, and dataset inclusion depends on automated quality checks.

Abstract

from arXiv · show

While large language models (LLMs) show promise in scientific discovery, existing research focuses on inference or feedback-driven training, leaving the direct modeling of the generative reasoning process, $P(\text{hypothesis}|\text{background})$ ($P(h|b)$), unexplored. We demonstrate that directly training $P(h|b)$ is mathematically intractable due to the combinatorial complexity ($O(N^k)$) inherent in retrieving and composing inspirations from a vast knowledge base. To break this barrier, we introduce MOOSE-Star, a unified framework that enables tractable and scalable training of $P(h|b)$, while supporting more scalable inference. In the best case, MOOSE-Star reduces complexity from exponential to logarithmic ($O(\log N)$) by (1) training on decomposed subtasks derived from the probabilistic equation of discovery, (2) employing motivation-guided hierarchical search to enable logarithmic retrieval and prune irrelevant subspaces, and (3) utilizing bounded composition for robustness against retrieval noise. To facilitate this, we release TOMATO-Star, a dataset of 108,717 decomposed papers (38,400 GPU hours) for training. Empirically, MOOSE-Star scales continuously with training data and inference budget, whereas direct brute-force sampling hits a complexity wall.

1. Introduction

Existing scientific-discovery research largely emphasizes inference or feedback-driven training rather than directly modeling P(h|b). MOOSE-Star addresses the resulting combinatorial intractability with decomposed training, hierarchical search, bounded composition, and a large processed-paper dataset.

  • Motivation: Most existing training approaches refine hypotheses using external feedback instead of modeling P(h|b) directly.These methods learn to update hypotheses given feedback, leaving the direct generative reasoning process underexplored.
  • The Complexity Barrier: Directly training P(h|b) is intractable because retrieving and composing k inspirations creates an O(N^k) search space.The paper frames this combinatorial complexity as the barrier to end-to-end training.
  • MOOSE-Star: MOOSE-Star operationalizes a probabilistic decomposition for training by separating discovery into sequential inspiration retrieval and hypothesis-composition subtasks.The framework converts the intractable objective into subtasks with linear complexity before applying further search reductions.
  • MOOSE-Star: Hierarchical search replaces full-literature scanning with top-down navigation, while bounded composition addresses imperfect retrieval through semantic tolerance.Together, these innovations target retrieval efficiency and robustness to noisy inspirations.
  • Resources: 108,717 processed papers were released in TOMATO-Star after approximately 38,400 GPU hours of computation.The dataset contains extracted research backgrounds, hypotheses, and historically linked inspirations.
  • Results: MOOSE-Star scales with training data and inference budget, whereas brute-force sampling reaches a complexity wall on multi-inspiration hypotheses.The framework is reported to provide scalable training and superior test-time scaling over brute-force sampling.

2. Related Work

Prior work has concentrated on inference algorithms and feedback mechanisms for scientific discovery, while comparatively few studies investigate training LLMs to model P(h|b) itself.

  • Training for Discovery: Most existing training efforts use external feedback, such as peer-review-derived critique models, rather than directly modeling P(h|b).These critique models provide reward signals for discovery agents.

3. Preliminary: The Decomposition Theory

The decomposition theory views a hypothesis as background combined with sequentially retrieved inspirations, then factorizes its probability into retrieval and composition steps.

  • Compositional View: A hypothesis h is modeled as the composition of research background b with k latent inspirations retrieved from a global knowledge base.The paper illustrates this view with scientific discoveries formed by combining existing backgrounds and concepts.
  • Probabilistic Decomposition: The intractable marginal likelihood P(h | b) is approximated by decomposing generation into k sequential steps under a Markov assumption.The intermediate hypothesis is treated as sufficient to summarize the history of prior inspirations.
  • Probabilistic Decomposition: Each factor represents either retrieving the next inspiration or composing the hypothesis with that inspiration.The intermediate states satisfy h0 = b and hk = h.

4. Dataset Construction

TOMATO-Star transforms scientific papers into structured background, hypothesis, and inspiration records through collection, decomposition, representation, and quality-assurance stages.

  • Pipeline: The dataset pipeline comprises collection, decomposition, structured representation, and quality assurance.These stages organize papers into components suitable for training the framework.
  • Data Collection and Split: 108,717 open-access papers span biology, chemistry, medicine, medical imaging, psychology, and cognitive science.The corpus uses a temporal split, with October 2025 papers held out for testing.
  • Preprocessing and Decomposition: Each paper is decomposed into a tuple of research background, hypothesis, and inspirations using locally deployed reasoning models.The workflow converts PDFs to Markdown before structured decomposition.
  • Structured Representation: Hypotheses are represented as incremental deltas, mapping each inspiration to one delta structured by motivation, mechanism, and methodology.The final hypothesis is formed by concatenating these deltas.
  • Quality Assurance: Every sample must pass checks for necessity, sufficiency, disjointness, and non-redundancy before inclusion.These checks require inspirations to be complementary, logically entail the hypothesis with the background, avoid leakage, and remain distinct.

5. Methodology

MOOSE-Star makes direct training of P(h|b) tractable by decomposing hypothesis generation, relaxing exact retrieval through bounded composition, and using hierarchical search for efficient retrieval.

  • Complexity barrier: O(N^k) direct training searches the Cartesian product of k inspirations, creating convergence difficulties at global-literature scale.For N ≈ 10^7 and k = 3, the search space is approximately 10^21.
  • Decomposed training: MOOSE-Star replaces monolithic modeling with sequential Inspiration Retrieval and Hypothesis Composition subtasks.Retrieval costs O(N), while composition given the exact inspiration has ideal relative complexity O(1), yielding a linear sum across k steps.
  • Bounded Composition: Bounded Composition expands the retrieval target from one exact inspiration to a semantic neighborhood of M candidates.This changes retrieval from O(N) to O(N/M) while increasing composition overhead from O(1) to O(M).
  • Bounded Composition: Because N ≫ M, shifting computation from global retrieval to local composition yields a net reduction in total complexity.The framework explicitly treats the increase in HC cost as a trade-off for reducing the dominant IR cost by a factor of M.
  • Bounded Composition: Semantic proxies operationalize bounded training by selecting Easy, Medium, and Hard candidates according to cosine-similarity ranges to the ground-truth inspiration.Candidates are retrieved from Semantic Scholar, embedded with SPECTER2, and stratified into three similarity tiers.
  • Hierarchical Search: Hierarchical Best-First Search ranks tree nodes with length-normalized path probabilities and expands the most promising semantic neighborhoods.In the best case, ideal routing makes search complexity proportional to tree depth, which is logarithmic in the search space.
  • Motivation Planning: MOOSE-Star introduces a Motivation variable derived from the background to guide the subsequent retrieval and composition stages.The motivation is designed as a lightweight approximately O(1) planning step.

6. Experiment

Experiments evaluate decomposed retrieval and composition training, bounded-composition robustness, and hierarchical search efficiency on the TOMATO-Star test set. The results show strong retrieval gains, improved composition, robustness to noisy inspirations, and substantially fewer search calls.

  • Experimental setup: October 2025 papers form the contamination-controlled TOMATO-Star test set used across the experiments.The evaluation section states that the test set comprises papers published in October 2025.
  • Inspiration Retrieval: 54.37% retrieval accuracy exceeds the 28.42% baseline after fine-tuning MS-IR-7B on 152k retrieval examples.The authors report checks indicating that the gain reflects genuine retrieval ability rather than superficial pattern exploitation.
  • Hypothesis Composition: MS-HC-7B significantly outperforms the baseline on ground-truth-input hypothesis composition under the GPT-4o-judged M3 rubric.M3 sums Motivation, Mechanism, and Methodology scores, with a maximum total of 12.
  • Overall results: Retrieval reaches near-frontier performance after targeted post-training, while composition improves substantially but remains clearly below frontier LLMs.The results indicate an asymmetry between the two decomposed subtasks.
  • Joint training: 54.34% IR accuracy and 5.02 HC performance show that MS-7B retains retrieval while improving composition over the separately trained models’ corresponding results.The reported comparison is 54.34% versus 54.37% for IR and 5.02 versus 4.68 for HC.
  • Robustness to noise: +0.22 total M3 score on Hard proxies is the largest gain for MS-7B over training with 2× bounded data.Bounded training improves scores across Easy, Medium, and Hard noise tiers, while multi-task training further strengthens robustness.
  • Hierarchical Search: 67.78 IR inference calls versus 218.00 for Tournament Search demonstrates roughly 3× lower search cost with a superior average rank.The hierarchical approach uses top-down, probability-guided pruning without compromising retrieval accuracy.
  • Motivation Planning: Detailed motivation planning achieves the best search efficiency among the compared motivation variants.The comparison uses simple background translation versus detailed directives derived from the delta hypothesis.

7. Scaling Analysis

MOOSE-Star’s decomposed subtasks scale with training data, while its hierarchical inference improves continuously as budget increases and brute-force sampling deteriorates with inspiration depth.

  • Brute-force training: 41×: BF pass rates collapse from 9.46% at k = 1 to 0.23% at k = 3, while decomposed subtasks remain approximately stable.BF must jointly sample all k innovations; decomposition makes k per-step decisions with bounded success rates.
  • Training-data scaling: Log-linear gains emerge for IR across 100 to 10^5 training samples, whereas HC gains appear only beyond 10^3 samples.The HC threshold is attributed to its higher data-density requirement as a generative task.
  • Generalization: Continuous log-linear IR scaling suggests the model acquires a generalizable “logic of discovery” rather than merely memorizing observed connections.This interpretation concerns out-of-distribution inspiration retrieval.
  • Test-time scaling: ∼6,000 calls: MOOSE-Star completes all 109 papers, while brute force saturates at ∼38.5% after ∼9,500 samples.The comparison uses the first 109 TOMATO-Star test papers and measures cumulative inference calls.
  • Test-time scaling: ∼53%: BF’s k = 1 win-or-tie rate falls to ∼37% at k = 2 and ∼8% at k = 3.Strict wins decline from 43% to 19% to 0% across the same depths.

8. Conclusion

The paper frames MOOSE-Star as a response to the combinatorial difficulty of directly modeling scientific hypothesis generation. Its framework combines decomposition, hierarchical search, bounded composition, and motivation planning, with scalable training and inference results.

  • Conclusion: MOOSE-Star addresses the combinatorial intractability of end-to-end P(h | b) modeling by operationalizing a probabilistic decomposition for training.The paper characterizes direct modeling as ill-posed because of combinatorial complexity.
  • Conclusion: The framework combines Decomposed Sequential Training, Hierarchical Search, Bounded Composition, and Motivation Planning.Together, these components reduce search complexity from exponential to logarithmic in the best case.
  • Conclusion: MOOSE-Star’s decomposed subtasks scale log-linearly with training data, while its test-time scaling remains continuous where brute-force methods hit a complexity wall.The complexity wall appears for hypotheses requiring multiple inspirations.

B. Theoretical Derivation: Rigorous Decomposition of P(h | b) via Hierarchical MDP

The derivation models hypothesis generation as a hierarchical sequential process: each step plans a motivation, retrieves an inspiration, and composes it into the evolving hypothesis. Under uniqueness and fixed-order assumptions, this factorizes P(h | b) into tractable stepwise terms.

  • Hierarchical formulation: MOOSE-Star represents each hypothesis as a sequence of motivation–inspiration pairs applied to the background.The motivation is a semantic strategy, while the inspiration is its concrete instantiation.
  • Assumptions: The uniqueness assumption assigns each valid hypothesis one unique minimal set of motivation–inspiration pairs.This collapses probability mass onto a single optimal ingredient set.
  • Assumptions: The fixed-order assumption selects a canonical integration sequence, avoiding marginalization over k! permutations.The assumption simplifies the generation process to one constructive order.
  • Hierarchical MDP: The hierarchical MDP uses the background and current hypothesis as state, with motivation as a high-level action and inspiration as a conditioned low-level action.Composition transitions the state by integrating the selected inspiration under the motivation.
  • Probabilistic decomposition: Equation 13 applies the chain rule across steps, then Equation 14 decomposes each step into planning, retrieval, and composition.The resulting terms are P(mj | b, hj−1), P(ij | b, hj−1, mj, I), and P(hj | b, hj−1, mj, ij, I).
  • Generalization: Relaxing fixed order requires marginalizing over all valid step permutations rather than using one canonical sequence.The generalized decomposition preserves the sequential structure while accounting for alternative valid orders.

C. Ablation of Inspiration Retrieval Experiment: Genuine Retrieval vs. Distribution Shortcuts

The retrieval ablation tests whether models exploit candidate-pool structure instead of matching query contexts to inspirations. Accuracy patterns across random, hard, and decoy negatives support genuine content-based retrieval.

  • Experimental design: The evaluation uses one positive and fourteen negatives, including approximately 35% hard negatives and 65% random negatives.The authors note that this shared construction could otherwise permit shortcut learning.
  • Negative construction: All Random substantially outperforms Original, with MS-IR-7B increasing from 54.37% to 78.58%.The result identifies hard negatives as the primary source of task difficulty.
  • Negative construction: All Hard remains close to Original, with MS-IR-7B at 53.42% versus 54.37%.Adding hard negatives beyond the original ∼35% ratio has diminishing effect.
  • Shortcut ablation: Decoy Cluster accuracies closely match All Random across models, indicating decisions rely on query–candidate content matching rather than related-candidate clusters.The decoy cluster is coherent but irrelevant to the current query context.
  • Conclusion: The authors conclude that MS-IR-7B’s original-benchmark performance reflects genuine inspiration retrieval rather than negative-construction artifacts.MS-7B shows the same pattern under multi-task training.

D. Additional Analysis of Decomposed Sampling Rates

Additional analyses show that decomposed hierarchical composition remains comparatively stable as inspiration depth grows, whereas brute-force sampling deteriorates sharply because it must satisfy all target innovations jointly. Context from the accumulated hypothesis state and background survey improves later-step composition, while repeated brute-force sampling remains vulnerable to hard-tail instances.

  • Decomposed Sampling Rates: BF per-dimension M3 scores drop sharply from single-step to multi-step cases, while HC scores remain comparatively stable across inspiration depth.Each BF attempt is scored against every target delta, whereas each HC attempt evaluates one assigned composition step.
  • Step-Position Analysis: Later HC steps show similar mean total scores but moderately higher pass rates, mainly on methodology, when conditioned on the accumulated previous-hypothesis state h_j−1.The step-position analysis treats j = 0 as unconditioned and later steps as conditioned on earlier deltas.
  • Controlled Ablation: Restoring h_j−1 raises HC pass rates to 37.9% at j = 1 and 44.2% at j = 2, versus 33.7% at j = 0; removing it yields 29.2% and 32.8%.The largest gains occur in methodology, while removing the background survey lowers absolute pass rates for both HC and BF.
  • Controlled Ablation: BF joint pass remains near zero at 0.45% without the background survey, whereas HC remains within a bounded per-step range of 22.5–24.6%.The survey supplies useful directional context, but the decomposition maintains bounded per-step success even when that context is removed.
  • Failure-Probability Perspective: Brute-force sampling can saturate despite non-trivial average pass rates because hard-tail instances with near-zero cumulative success remain unsolved under practical budgets.Repeated-sampling behavior depends on the distribution of instance-level failure probabilities, not only the average single-attempt rate.

F.1. Quantitative Evidence on Temporally Out-of-Distribution Data

The temporal out-of-distribution evaluation tests whether models trained on pre-October 2025 literature can retrieve inspirations and reconstruct hypotheses from October 2025 papers. Retrieval is the main bottleneck, while stronger composition backends substantially improve success once the correct inspiration is available.

  • Evaluation Setting: The evaluation uses October 2025 papers as a strict temporal OOD test, with models fine-tuned exclusively on pre-October 2025 data.This separates the held-out evaluation literature temporally from the training data.
  • Evaluation Criteria: Temporal reconstruction success requires both retrieving the ground-truth inspiration within top-K and producing a conditioned hypothesis meeting the target M3 score.The evaluation uses 200 test samples and an inspiration corpus of 3,035 candidates.
  • Model Comparison: 17.5% versus 15.0% at Rank ≤50, M3 ≥5, with multitask MS-7B outperforming single-task MS-HC-7B.MS-7B is jointly trained on inspiration retrieval and hypothesis composition data.
  • Retrieval Results: Retrieval succeeds for 20.0% of cases within Rank ≤25, 28.0% within Rank ≤50, and 29.0% within Rank ≤100.The small increase from Rank ≤50 to Rank ≤100 indicates that correct inspirations are rarely ranked beyond the top 50 in this evaluation.
  • Composition Results: Under Rank ≤50, MS-HC-7B reaches 4.0% success at M3 ≥7, compared with 13.5%–21.0% for stronger frontier composition backends.The comparison indicates that composition quality remains consequential after the relevant inspiration is identified.
  • Two-Inspiration Case Study: In the k = 2 TREM2 case, MOOSE-Star ranks the two ground-truth inspirations at 12/3035 and 1/3035, respectively.The case study illustrates sequential retrieval and composition of two inspirations in a temporally unseen paper.

G. Limitations

The paper’s limitations concern scope: it targets trainability of the hypothesis proposal distribution rather than feedback-driven refinement, and evaluates literature-derived reconstruction rather than prospective experimental discovery.

  • Scope: The work does not study feedback-driven refinement or experiment-guided ranking after hypotheses have been proposed.Its scope is making P(h | b) tractably trainable before external feedback is available.
  • Empirical Boundary: Prospective experimental testing of newly proposed hypotheses is not included in the present study.Experiments instead assess recovery of discovery-relevant inspirations and reconstruction of corresponding hypotheses in temporally held-out literature-derived cases.
Loading 2603.03756v4…