Training Reasoning Models to Generate Diverse Scientific Ideas

• Existing LLM ideation typically generates and judges one idea at a time, rather than comparing candidates within a shared set.Source: §1 · p. 1
• With independently sampled candidates, diversity is incidental rather than deliberately designed, a shortcoming also noted for autonomous research agents.Source: §1 · pp. 1–2
• The proposed approach treats ideation as creative search, generating K source-grounded ideas whose quality is weighted by their distance from earlier ideas.Source: §1 · p. 2
• Training first uses supervised fine-tuning to initialize the policy, then GRPO optimizes the set-level objective.Source: §1 · p. 2
• The trained 8B model achieves the highest creative utility on the test set; pilot experts find its ideas comparably promising to Claude Opus 4.8, and sandbox success@k is comparable.Source: §1 · p. 2

2.1 BACKGROUND

• GRPO samples multiple rollouts and updates the policy using each rollout’s reward relative to the group mean, without a learned critic.Source: §2.1 · pp. 2–3

2.2 SET-LEVEL OBJECTIVE

• Each prompt yields K ideas in one generation, treated as one rollout and scored with a single scalar reward and shared advantage.Source: §2.2 · p. 3
• Each idea’s quality is multiplied by its distance from the nearest earlier idea, then the item rewards are aggregated into a set reward.Source: §2.2 · p. 3
• The objective combines an output-quality score with a pairwise distance measure, and can apply wherever a task has quality and diversity signals.Source: §2.2 · p. 3
• Unlike evaluation-time Creative Utility, training omits its patience discount and uses generation order rather than the best ordering of the set.Source: §2.2 · p. 3

3 DOCUMENT GROUNDED IDEATION

• Each idea is a structured card with a problem gap, core claim, method, and an inspiration field naming the source documents and how they inform it.Source: §3 · p. 3

3.1 REWARD INSTANTIATION

• Quality represents long-term scientific impact, judged pairwise against a gold idea; soundness is a separate axis probed only in a small execution check.Source: §3.1 · p. 4
• Distance averages LLM-judged semantic distances across the problem-gap, claim, and method fields, using equally weighted scores derived from a 1–5 similarity rubric and normalized to [0, 1].Source: §3.1 · p. 4

3.2 DATA

• The data pipeline selects source documents and a held-out gold idea from published papers; training focuses on Language, while the 2026 test set spans 16 domains.Source: §3.2 · p. 4
• A teacher model constructs SFT examples with idea sets and hindsight-generated reasoning traces; the training and test splits contain 7,172 and 1,740 instances, respectively.Source: §3.2 · p. 4

3.3 TRAINING DETAILS

• Training uses Qwen3-8B with K = 3, first applying supervised fine-tuning and then GRPO in think and no-think variants.Source: §3.3 · pp. 4–5

• Baselines include closed-source Claude Opus 4.8 and GPT-6 Astra, Nemotron-120B, and retrained competing objectives that isolate diversity and the SFT warm-start.Source: §4 · p. 5
• The three benchmarks are a contamination-free 1,740-instance test set, GiantsBench’s 2,374-instance held-out split, and a 500-instance IdeaSeed sample.Source: §4 · p. 5
• Creative Utility is the primary metric; the evaluation also reports length-controlled impact, mean pairwise distance, grounding, and IdeaSeed idea-type entropy and total variation distance.Source: §4 · pp. 5–6
• Table 1 reports that Proposed (no think) has the highest creative utility, while Proposed (think) is on par with DARLING and the no-SFT ablation.Source: Table 1 · p. 6

5 RESULTS

• 0.64 CU for Proposed (no think) exceeds Claude Opus 4.8 (0.38) and GPT-6 Astra (0.28), with high win-rate and distance between ideas.The thinking variant reaches 0.51 CU, on par with DARLING and the no-SFT ablation, but below Proposed (no think).Source: §5 · p. 6
• 0.88 average distance for Proposed (no think) exceeds DARLING’s 0.67; Impact-only RL reaches 43% win-rate but collapses the set to 0.38 average distance.The results indicate that rewarding diversity within a generated set is more effective than enforcing it across independent rollouts.Source: §5 · p. 6
• Grounding is 0.81 with SFT versus 0.19 without it, while the no-SFT model remains 0.13 CU below Proposed (no think).The authors attribute the grounding difference to the SFT warm-start, which lets the set-level reward raise utility without losing grounding.Source: §5 · p. 6, Figure 2 · p. 7
• On GiantsBench, Proposed reaches 68.3% win-rate against the gold idea versus GIANTS-4B’s 18.3%, and average pairwise distance is 0.939 versus 0.025.The table reports results on 2,374 test papers.Source: Table 2 · p. 7, §5 · p. 7
• On IdeaSeed’s Method Paradigm axis, Proposed (think) reaches 0.73 (TVD 0.30), while Proposed (no think) reaches 0.65 (TVD 0.43).Human proposals reach Hnorm = 0.87; the authors note that their objective targets diversity within sets, not spread of idea types across instances.Source: Table 3 · p. 7, §5 · p. 7
• Appendix E reports generalization to a different model family, per-domain results, and changes in win-rate and diversity by idea position.Source: §5 · p. 7

6.1 HUMAN STUDY: PORTFOLIO QUALITY AND DIVERSITY

• Four NLP PhD experts each evaluated 15 instances, judging promise, substantive uniqueness, and uniqueness among ideas judged promising.They assessed blinded sets from Claude Opus 4.8, Qwen3-8B, and Proposed (no think); ideas counted as the same direction when carrying out one would largely address the other.Source: §6.1 · p. 8
• Experts judged the model’s portfolios to contain 2.17 promising ideas and 2.83 unique ideas on average, versus Claude Opus 4.8’s 2.20 and 2.72.The authors characterize the two systems as similarly promising; Table 4 aggregates judgments from four annotators across 60 instances.Source: Table 4 · p. 8, §6.1 · p. 8

6.2 CAN IDEAS BE EXECUTED?

• For execution evaluation, 15 instances supplied sets of three ideas, each frozen into a card and tested independently by Claude Code in an identical sandbox.The sandbox imposed a 90-minute, $30 budget and no GPU; the agent designed and ran a proof-of-concept test and reported supports, refutes, or inconclusive.Source: §6.2 · p. 8
• Across tested k values, the models are comparable to Claude Opus 4.8 on success@k, with no evidence of a quality loss relative to Claude.Table 5 reports execution results for 132 ideas; success@k counts instances with at least one idea receiving a supports verdict, or averages across subsets when k < 3.Source: Table 5 · p. 8, §6.2 · p. 8
• A qualitative review found that one coding-agent supports verdict may be stronger than its evidence, so the authors treat verdicts as early signals for researcher follow-up.In the example, the agent replaced a probe that could not test one claim conjunct with three pre-registered probes and fixed decision rules before running them.Source: §6.2 · p. 9

7 RELATED WORK

• Quality-diversity methods commonly reward diversity across independent rollouts or select diverse preference pairs, while newer set-level methods require a known answer set or fixed reward space.Open-ended ideation lacks either because distinctness is judged in the context of the other ideas in a set.Source: §7 · p. 9
• Literature-grounded systems use revision, critique, or clustering after candidates are generated, while GIANTS and Tong et al. train one-idea-at-a-time systems.The authors contrast these approaches with training the generator itself to produce ideas that differ from one another.Source: §7 · p. 9
• Execution-reward ideation has been reported to collapse onto a few simple ideas; this work instead generates a set from a whole body of documents and scores each idea against earlier ideas in that set.Execution is used to evaluate the resulting sets downstream.Source: §7 · p. 9

8 CONCLUSION

• The best 8B model without reasoning achieves higher creative utility than strong closed-source models and competing quality-diversity objectives; the reasoning variant beats closed-source models and is on par with DARLING.The paper reports comparable sets in small pilot human and execution studies, with no evidence that diversity costs quality.Source: §8 · p. 9
• The set-level objective generates K ideas in one rollout and weights each idea’s quality by its distance from ideas emitted earlier.The objective is adapted for document-grounded ideation and balances diversity with idea quality.Source: §8 · p. 9
• Future work includes grounding-aware rewards, diversity across a population, and reasoning traces designed for creative ideation.Source: §8 · p. 9

LIMITATIONS

• The study focuses on scientific ideation using context and parametric knowledge rather than retrieval.The authors identify retrieval as a direction for future work.Source: LIMITATIONS · p. 10
• The authors frame the current setup as testing set-style generation aimed at increasing diversity while also improving quality.Source: LIMITATIONS · p. 10
• Training is limited to English-language natural language processing scientific papers.Source: LIMITATIONS · p. 10

AI USE STATEMENT

• Claude Opus-4.8 generated the training data, and generative AI also assisted with analysis and revisions; the hypotheses and experiment design were human-originated.Source: AI USE STATEMENT · p. 13

A.1 DATA FORMAT

• Each idea card is a JSON object with a problem gap, core claim, method, and inspiration field.The gap identifies an unresolved connection across sources; the claim is a testable assertion, the method sketches an approach, and inspiration explains the source relationship and leap.Source: §A.1 · p. 13

A.2 DATA CURATION: GATHERING DOCUMENTS

• The dataset pairs a target paper’s gold idea with selected core references, which serve as method inputs.Targets come from the GiantsBench training split; source references are classified by their role in motivating or informing the target, then summarized into source cards.Source: §A.2 · pp. 13–14
• Targets with two or fewer source documents or no gold idea are dropped before the sources and target are joined into one record.Table 6 illustrates source-reference labels using citation context and the assigned functional role; Figure 3 reports retained-reference counts and source-type distributions.Source: §A.2 · p. 14, Table 6 · p. 14

A.3 TRAINING DATA CURATION

• The training pipeline has two phases: supervised fine-tuning followed by GRPO.Source: §A.3 · p. 14
• The SFT data pipeline generates K ideas per instance with Claude Opus-4.8, then creates hindsight thinking traces for those ideas.The traces include exploring context, checking candidates, and ensuring ideas are distinct and correctly tagged.Source: §A.3 · p. 15
• Completions average 4,000 tokens, comprising 1,300 tokens for three ideas and 2,800 tokens for the thinking trace.The figure reports distributions of input-document counts, input tokens, and publication years across train and test splits.Source: §A.3 · p. 15
• Training uses the language domain, while testing spans different domains for out-of-domain generalization.Figure 4’s full pools contain 8,007 training and 1,741 test instances after source-count filtering; 7,172 training instances are used for SFT and 1,740 for evaluation.Source: §A.3 · p. 15, Figure 4 · p. 15

B TRAINING DETAILS

• The same prompt template is used at training and inference, with K set to 3 in the main runs and source documents supplied as a numbered block.The section introduces metrics for impact, distance, grounding, and creative utility.Source: B TRAINING DETAILS · p. 15, C METRICS · p. 18
• The reward adds a schema-validity gate and bonus, plus a capped penalty for overly long idea cards.Invalid rollouts receive zero and skip judging; valid rollouts receive a 0.05 bonus. The penalty applies above 1,000 tokens and is capped at 3.Source: B TRAINING DETAILS · pp. 15–16

B.1 HYPERPARAMETERS

• Training runs SFT then GRPO; each GRPO step samples 12 prompts with four rollouts, each containing K=3 ideas scored for quality and distance.The reported proposed-model checkpoints are at step 600; the same recipe is used for the Olmo-3 runs.Source: §B.1 · p. 16

B.2 SFT VARIANTS: WITH AND WITHOUT THINKING

• The two SFT variants share identical answer cards and data, differing only in whether the teacher-generated reasoning trace is included.Both use 7,172 training instances and 100 validation instances before the same GRPO stage.Source: §B.2 · pp. 16–17

B.3 DARLING

• DARLING enforces diversity across separate single-idea rollouts, whereas this method enforces it within one generated set.The reimplementation uses eight rollouts per prompt and a cluster-based diversity score; three independent test-time generations form its evaluation set.Source: §B.3 · p. 17

B.4 OLMO-3 DETAILS

• The recipe is also applied to Olmo-3-7B-Think-SFT, using the same SFT and GRPO settings as the main run.The length penalty uses the Olmo tokenizer, whose card-text counts differ from Qwen3 by about 0.5%.Source: §B.4 · p. 17

B.5 COMPUTE

• A GRPO step takes about one minute, so an approximately 600-step epoch takes roughly 10 hours on the stated eight-H200-GPU setup.SFT takes about 46 minutes per epoch on 7,172 instances.Source: §B.5 · p. 17

C.1 QUALITY

• Quality is the pairwise impact win against a gold idea, judged in both presentation orders and scored as 0, 0.5, or 1.On 122 real-world paper pairs, GPT-5.4-mini selects the higher-impact idea 74% of the time and SciJudge-30B 65%.Source: §C.1 · p. 18

C.2 DISTANCE

• Pairwise distance averages equally weighted, inverted similarity scores across the problem gap, core claim, and method, excluding inspiration.On 100 within-set pairs, the training judge’s aggregated-distance agreement with humans is ρ=0.37 versus human–human ρ=0.36; the evaluation judge reaches ρ=0.50.Source: §C.2 · p. 19, Table 10 · p. 19

C.3 GROUNDING

• Grounding measures whether an idea depends on its cited sources, but it is tracked as a diagnostic rather than included in the training reward.The authors report that grounding can fall as ideas move away from sources, and that the post-RL drop is large because the reward does not penalize drift.Source: §C.3 · pp. 19–20

C.4 CREATIVE UTILITY

• Creative utility orders ideas by quality first, then rewards each later idea for quality multiplied by its distance from the closest selected idea.The reported greedy farthest-point ordering is a lower bound on the best ordering and is independent of the model’s writing order; reported CU uses γ=0.9 and is normalized.Source: §C.4 · p. 20
• Training reward uses generation order without a patience discount, while evaluation CU uses the best ordering, a discount, normalization, and evaluation judges.The paper specifies these as differences between Rset and CU.Source: §C.4 · p. 20

GiantsBench

• For GiantsBench, the authors use their training prompt for their models and GIANTS-4B’s in-distribution prompt, comparing systems at the claim level.Source: D BENCHMARKS · p. 20

IdeaSeed

• The IdeaSeed evaluation uses 500 papers, sampling 250 per domain after excluding papers with fewer than three related works.This filtering makes each retained instance require synthesis across several source documents.Source: D BENCHMARKS · pp. 20–21
• IdeaSeed diversity is measured with normalized label entropy, total variation distance, and Jensen–Shannon divergence.Higher entropy means a more even spread across types; lower TVD and JSD mean a distribution closer to human proposals.Source: D BENCHMARKS · p. 21

Results

• The proposed model has the highest win-rate and number of instances with at least one win; some competing objectives have similar win-rate but lower utility from diversity collapse.Table 11 covers 1,740 test instances and also reports grounding.Source: E RESULTS · p. 21

Sampling vs Set Level Prompting

• Set-level prompting raises distance but can reduce quality; the authors attribute the proposed model’s major gains to reward training rather than prompting alone.For Claude Opus 4.8, distance rises from 0.24 to 0.80 while win-rate falls from 0.33 to 0.21.Source: E RESULTS · p. 21, Table 12 · p. 22

Performance across different domains

• Across the 16 test domains, the model maintains high creative utility, with a dip for Proposed (think) on Mathematics.Source: E RESULTS · p. 21, Figure 6 · p. 22

Transfer of recipe to other models

• The proposed objective and recipe also generalize to Olmo-3-7B-Think-SFT on the contamination-free test set.Table 14 presents the transfer results across 1,740 instances.Source: E RESULTS · p. 22, Table 14 · p. 23

Grounding Trade-Off

• As ideas proceed through a generated set, grounding declines; the decline is more prominent for the no-think model, while the think model puts its most impactful idea first.Source: E RESULTS · p. 22, Table 15 · p. 23

Think vs no think

• The no-think variant has higher utility than the think variant, but the forward-full trace experiment does not beat the no-think baseline.The authors note that filtering for high-quality, diverse sets left around 1,000 SFT instances, making training-data structuring difficult to isolate from data size.Source: E RESULTS · pp. 22–23

Human study

• The human study assigns portfolios to annotators whose selected expertise areas match the target papers’ tagged subareas.Table 13 reports results broken down by target-paper subarea; the supplied results note that Claude leads on promising ideas in three areas while ours has the most or tied-most unique ideas in five of six.Source: E RESULTS · p. 23, Table 13 · p. 23

IdeaSeed Benchmark

• On IdeaSeed, prompted baselines have higher entropy on the Opportunity axis than either proposed variant, suggesting RL may reduce population-level type diversity.The reported values are 0.89 for Claude Opus 4.8, 0.80 for Qwen3-8B, 0.67 for Proposed (no think), and 0.75 for Proposed (think).Source: E RESULTS · p. 24, Table 17 · p. 24

F AUTORESEARCH

• The proposed open, modular benchmark tests content fidelity, contrastive retrieval, interpretability, and stability across memory mechanisms to identify which store information and where designs share failures.The proof of concept used HotpotQA and NQ-Swap, seven CPU-runnable mechanisms, and two null controls; the benchmark aims to map performance by subtask and support reproducible comparisons.Source: F AUTORESEARCH · p. 25
• The first run separated real mechanisms from null memory but did not establish a shared failure: NQ-Swap scores were 0.87–0.96 for dense encoders, and only two of seven arms failed.Noise at σ = 0.01–0.02 was also undetectable, so the agent revised its plan to add entity-order, negation, and multi-item-capacity probes and widen the stability sweep.Source: F AUTORESEARCH · pp. 26–27
• Spearman 0.96 on retrieval across disjoint data halves and 0.18 between fidelity and retrieval across seven real arms accompanied a supports verdict; argument-order insensitivity spanned every tested design family.The authors’ check was informal, and they caution that the evidence may not fully support the verdict; “supports” means the benchmark design is worth building, not that its claim is established.Source: F AUTORESEARCH · pp. 27–28

G PROMPTS

• The reference-selection prompt defines sparkers as works that motivate the paper, expose a targeted gap, provide its conceptual or methodological foundation, or are directly extended.It instructs the model to read all mentions of each reference together, apply every fitting label, and distinguish central from incidental influence.Source: G PROMPTS · pp. 29–30
• The prompt excludes tools, datasets, and generic context citations without a load-bearing role, requires a self-consistency check, and assigns every reference to either sparkers or other ids.For each sparker it requests a short rationale and verbatim context quotes; the worked examples distinguish substantive motivation or gap evidence from decorative mentions.Source: G PROMPTS · p. 30
• The reference prompt asks the model to classify each reference’s role using section context and a restricted set of labels.It distinguishes sparking references from the larger set of other references and asks the model to assess each reference-context pair.Source: G PROMPTS · p. 30
• The source-paper summarization prompt requires five fields, separating the conceptual gap, claim, and method from empirical results and takeaways.It asks for concrete reported numbers and negative or null findings where available, and reserves results and metrics for the outcome fields.Source: G PROMPTS · p. 31
• The target-paper prompt recovers one core idea grounded in supplied inspiration documents, with labeled gap and method fields and a forward-looking proposed approach.Its inspiration field must identify each relied-on document’s limitation and explain why connecting the documents is non-obvious; the prompt prohibits grounding the idea in the target paper’s own bibliography.Source: G PROMPTS · p. 32
• The hindsight-trace prompt requires exploratory reasoning that reaches the final idea set through overshooting, cutting or merging candidates, and revisiting judgments.It also requires uncertainty, at least one label correction, verification against source documents, and honest treatment of weak or routine inspiration links, while forbidding a final audit-style recap.Source: G PROMPTS · pp. 33–35
• A proposed task-level weight schedule should down-weight hard tasks early and up-weight them late, unifying curriculum and focal policies as limiting cases.The proposal tests whether this non-monotonic schedule outperforms either fixed policy, using a distinguishability score and progress-dependent exponent.Source: G PROMPTS · p. 38
• The section proposes testing whether supervised contrastive pretraining with all same-relation statements as positives yields more separable FewRel prototypes than MTB or cross-entropy baselines.The proposed synthesis combines SupCon’s multi-positive loss with MTB’s relation encoder and blank-replacement trick, then evaluates embeddings with prototypical networks.Source: G PROMPTS · p. 38
• Per-episode distinguishability scores are proposed as predictors of prototype failure, with k-prototypes expected to help only in low-distinguishability episodes.The proposed empirical mapping correlates hardness scores with prototypical-network error and compares single- versus k-prototype representations on miniImageNet and FewRel.Source: G PROMPTS · p. 38
• The ideation prompt requires structured cards separating the problem gap, testable core claim, proposed method, and critical document-based inspiration, while encouraging distinct ideas.It also directs the model to name source limitations, justify non-obvious connections, and order the strongest idea first.Source: G PROMPTS · pp. 39–40
• Pairwise similarity is scored separately for whether ideas target the same research gap, using prose rather than shared taxonomy labels.The prompt uses a 1–5 scale, where identical labels alone indicate at most weak similarity.Source: G PROMPTS · p. 41
• The similarity judge separately scores whether ideas make the same central, testable claim, distinguishing partial overlap from equivalent claims.The prompt defines higher similarity as claims where confirming one would largely confirm the other.Source: G PROMPTS · p. 41
• Method similarity depends on whether ideas use the same testing paradigm and concrete instrument, not merely a shared broad tool or label.The 1–5 scale distinguishes related designs from approaches that would be executed nearly identically.Source: G PROMPTS · p. 41
• A grounding judge rates whether an idea’s insight depends on the specific cited findings or would stand unchanged if those documents were removed.It is instructed to distinguish concrete, load-bearing grounding from decorative citation language.Source: G PROMPTS · p. 42