PaperlayerView exact revision on arXiv

Source-linked AI summary

AI Can Learn Scientific Taste

Jingqi Tong, Mingzhe Li, Hangcheng Li, Yongzhuo Yang, Yurong Mou, Weijie Ma, Hongji Chen, Xiaoran Liu, Qinyuan Cheng, Ming Zhang, Qiguang Chen, Weifeng Ge, Qipeng Guo, Tianlei Ying, Tianxiang Sun, Yining Zheng, Zhiheng Xi, Xinchi Chen, Jun Zhao, Ning Ding, Xuanjing Huang, Yu-Gang Jiang, Xipeng Qiu

arXiv:2603.14473v3cs.CL

TL;DR

Scientific taste—the ability to judge and propose high-impact research—remains difficult for AI systems and is concentrated among experienced specialists. The paper introduces RLCF, using citation-based community feedback to train Scientific Judge and Scientific Thinker. Scientific Judge outperforms strong LLM baselines and generalises across time, metrics, and fields, while Scientific Thinker generates ideas judged to have higher potential impact than baseline ideas.

  • Problem

    Scientific taste is scarce among specialised human experts, and whether AI can learn judgement and high-potential research ideation remains an open question.

  • Method

    RLCF uses citation signals as community feedback to train Scientific Judge for paper comparison and Scientific Thinker for high-potential research ideation.

  • Results

    Scientific Judge outperforms strong LLM baselines and generalises to future-year papers, other community metrics, and unseen fields; Scientific Thinker proposes higher-potential-impact ideas than baselines.

  • Takeaways & Limitations

    The results suggest that AI can learn scientific taste from community feedback, supporting AI systems intended to help accelerate scientific discovery.

  • Takeaways & Limitations

    Scientific taste may extend beyond judgement and ideation, and citations are imperfect feedback because influential papers can receive delayed recognition and citation-based evaluations retain biases.

Abstract

from arXiv · show

Scientific discovery depends on expert judgement and foresight, which we call scientific taste: the ability to judge and propose research ideas with the potential for long-term scientific impact. Scientific taste is largely concentrated among highly experienced researchers, whose expertise is usually limited to a few specialised fields. If AI could learn scientific taste, it could reduce reliance on human experts and accelerate scientific discovery. Whether AI can learn this ability remains an open question. We introduce Reinforcement Learning from Community Feedback (RLCF) to learn judgement and ideation. Scientific Judge learns from community feedback, such as citations. Scientific Thinker learns to propose research ideas with high potential impact. Experiments show that Scientific Judge outperforms strong LLM baselines and that learned judgement generalises to future-year papers, other community metrics, and unseen fields. Furthermore, Scientific Thinker proposes research ideas with higher potential impact than those proposed by baselines. These results suggest that AI can learn scientific taste, marking an important step towards AI systems that could help accelerate scientific discovery.

1 Introduction

Scientific taste is the expert ability to judge and propose research with long-term impact, but it is scarce and specialised. The paper proposes learning this ability from community feedback and reports gains in judgement and ideation.

  • Scientific taste combines judgement of genuine advances with foresight about promising follow-up research directions.
  • Community verdicts about scientific value emerge through long-term researcher interactions and are reflected in signals such as citations.
  • Scientific taste is scarce because it depends on knowledge, skill, and sustained experience, while researchers typically specialise in limited areas.
  • RLCF trains Scientific Judge to compare papers using community feedback and Scientific Thinker to propose ideas guided by learned judgement and potential impact.
  • Scientific Judge outperforms strong LLM baselines, generalises across future years, community metrics, and unseen fields, and Scientific Thinker proposes higher-impact ideas than baselines.

2 Background and Related Work

The paper formalises scientific taste as judgement plus high-impact ideation and distinguishes it from AI systems focused mainly on executing research. It motivates RLCF as a community-supervised alternative for open-ended scientific evaluation and generation.

  • Definition of Scientific Taste: Potential impact is formalised through cumulative expected future citations, with larger I(p) indicating higher potential impact.
  • Definition of Scientific Taste: JudgeCap measures expected accuracy when comparing the cumulative expected impact of field- and time-matched paper pairs.
  • Definition of Scientific Taste: ThinkerCap measures the expected impact of ideas generated from seed papers, and one model is stronger when its ThinkerCap exceeds another’s.
  • Definition of Scientific Taste: Scientific taste is formally defined as the combination of judgement capability and ideation capability, requiring both high JudgeCap and high ThinkerCap.
  • AI for Scientific Research: Existing AI Scientist training targets literature search and experiment execution, leaving judgement of valuable directions and high-impact ideation comparatively underdeveloped.
  • RL Training Paradigms for LLMs: RLCF uses naturally occurring community feedback rather than costly individual annotations to capture community-level preferences for open-ended scientific tasks.

3 Reinforcement Learning from Community Feedback

RLCF converts community signals into preference supervision, trains Scientific Judge with GRPO, and uses it to reward Scientific Thinker through pairwise comparisons. This design supports scalable judgement and open-ended ideation training.

  • RLCF has three stages: constructing community preferences, training Scientific Judge as a preference model, and aligning a policy with its rewards.
  • Citations provide community feedback, while same-field, same-year article pairs reduce field and time biases in preference data.
  • SciJudgeBench represents scientific ideas by titles and abstracts with binary labels indicating which paired item has higher relative citations.
  • Preference Modeling: Scientific Judge: Scientific Judge uses GRPO to sample reasoning-and-preference outputs and receives binary correctness rewards based on observed pairwise labels.
  • Preference Modeling: Scientific Judge: The Judge policy maximizes a clipped surrogate objective with a KL penalty toward a reference policy, using normalized within-group advantages.
  • Preference Alignment: Scientific Thinker: Scientific Thinker uses Scientific Judge as a generative reward model, ranking sampled ideas through round-robin pairwise comparisons and rewarding their group win rates.

4 AI Can Learn Scientific Judgement

Scientific Judge learns scientific judgement through reinforcement learning from community feedback, with performance scaling across data and model size. Its learned judgement generalises to future papers, unseen fields, other community metrics, and controlled comparisons.

  • Training and evaluation: Scientific Judge is trained on SciJudgeBench, constructed from 2.1M arXiv papers and 720,341 field- and time-matched citation-preference pairs.The models compare papers using citation-derived preference labels.
  • Scaling trends: Scientific Judge learns scientific judgement effectively across model scales and series, revealing scaling behavior with both data amount and model size.Training uses preference prediction correctness as a verifiable reward with GRPO.
  • Scaling trends: 58.1 to 77.3 and 69.7 to 82.7: overall scores rise from untrained base models to final Qwen3-4B and Qwen3-30B checkpoints, respectively.The gains are observed across all fields and follow an approximately log-linear relationship with training-data scale.
  • Scaling trends: 61.9 to 81.5: average accuracy increases across the Qwen2.5 family as model size grows from 1.5B to 32B, while Qwen3-30B reaches 82.7 versus 77.3 for Qwen3-4B.The final SciJudge-Qwen3-30B checkpoint surpasses all listed strong LLM baselines in average accuracy.
  • Generalization: 80.9 and 83.1: on 2025 papers, Qwen3-4B and Qwen3-30B improve from 64.7 and 71.7, respectively, after RLCF training.The gains persist for papers published after the corresponding base models were released.
  • Generalization: Citation-trained judgement transfers across unseen fields and community metrics, while controlled comparisons show gains of 8.3 to 11.8 percentage points.Cross-field gains include 58.1 to 68.9 for Qwen3-4B and 69.7 to 76.4 for Qwen3-30B; metric transfer covers review scores and Altmetric attention scores.

5 AI Can Learn Ideation with High Potential Impact

Scientific Thinker uses Scientific Judge as a reward model to train policies that propose research ideas with higher potential impact. The gains persist on later seed papers, outperform a baseline reward model, and make the 30B policy competitive with strong LLMs.

  • Method: Scientific Thinker is trained with Comparison-Based GRPO using Scientific Judge as the reward model, on 4,000 high-citation papers from 2025.Two policy sizes are evaluated on 200 in-domain and 200 out-of-domain seed papers.
  • Ideation results: 81.5% and 76.5%: SciThinker-30B and SciThinker-4B achieve these in-domain win rates against their base policies after training with SciJudge-Qwen3-4B.Win rates are judged by majority vote from three strong LLMs assessing potential impact.
  • Ideation results: 83.0% and 76.0%: the 30B and 4B policies retain ideation gains on seed papers published after the training period.This evaluates temporal transfer to an out-of-domain holdout.
  • Reward-model comparison: 81.5% versus 73.0% and 83.0% versus 70.5%: Scientific Judge outperforms Qwen3-4B-Instruct as a reward model on in-domain and out-of-domain 30B evaluations.The same ordering holds for both policy sizes.
  • Takeaway: These experiments suggest that reinforcement learning can improve both scientific judgement and ideation with high potential impact.The reported ideation gains transfer to seed papers published after training.

6 Conclusion

The paper concludes that AI can learn scientific taste from large-scale community feedback, using citation-informed judgement and ideation to support potential acceleration of scientific discovery.

  • Scientific Judge learns scientific judgement from community feedback, while Scientific Thinker uses that judgement to propose high-impact research ideas.The approach uses citation signals for preference modeling and alignment, then applies Scientific Judge as a generative reward model for ideation.
  • Scientific Judge generalises to future-year papers, other community metrics, and unseen fields, while retaining gains under controlled contextual comparisons.The controlled comparisons match author and institutional context, publication time, and topic similarity.
  • Scientific Thinker proposes research ideas judged to have higher potential impact than ideas from strong LLM baselines.
  • Overall, the results suggest that AI can learn scientific taste from large-scale community feedback and may help accelerate scientific discovery.

Limitations and Future Work

The paper identifies limitations in its formulation of scientific taste, its citation-based feedback, its ideation evaluation, and the paper context used by Scientific Judge.

  • Scientific taste may involve more than judging and proposing high-impact ideas, including assessing experimental feasibility and recognising distinctive, diverse ideas.
  • Citations are imperfect community feedback because some high-potential papers receive few citations initially and become influential only later.Citation dynamics and finer-grained field clustering are proposed as future improvements, while citation-based biases cannot be fully removed.
  • Scientific Thinker’s ideation evaluation mainly relies on strong LLM evaluators, and proposed ideas are not experimentally validated.The paper notes that this evaluation may not fully reflect the ideas’ potential impact.
  • Scientific Judge is trained mainly on titles and abstracts, so incorporating richer context such as related-work sections may improve judgement.

Ethical Considerations

The work uses public paper metadata and citation-based procedures, while acknowledging that community signals may encode biases across fields or topics.

  • The study uses publicly available paper metadata and does not involve private or sensitive information.
  • Citation-based signals may encode biases across fields or topics, so preference pairs are constructed within matching subcategories and similar publication windows.
  • The evaluation includes controls matching author, institution, publication time, and topic similarity to reduce simple contextual explanations.These controls constrain comparisons but do not remove all biases in citation-based community feedback.
  • The paper states that systems using these signals should assist rather than replace human judgement in scientific evaluation.

A.3.2 Metric OOD Test Sets

The paper tests transfer beyond citation-based in-domain evaluation using peer-review and attention metrics, matched-context controls, unseen fields, and multiple model families.

  • Metric OOD Test Sets: ICLR mean reviewer ratings define preferences for testing transfer from citation-based training to peer-review evaluation.Confidence and rating variance are used only for quality filtering.
  • Metric OOD Test Sets: Altmetric attention scores test transfer from citation preferences to a distinct online-attention signal rather than scientific quality or societal impact.Altmetric aggregates attention from social media, news, policy documents, patents, and blogs.
  • Control Test Sets: Author- and institution-matched pairs test whether citation-derived preferences remain informative when papers share a CSRankings faculty member and publication context.Pairs share publication quarter and first-two arXiv subcategory while requiring citation differences of at least 12 and 50% relative difference.
  • Control Test Sets: Topic-control pairs require shared field, subcategory, and year plus cosine similarity of at least 0.70, isolating comparisons with tighter semantic matching.
  • Field OOD: Cross-field transfer includes 160 bioRxiv pairs entirely absent from training; Qwen3-4B and Qwen3-30B reach 56.3% and 70.6% accuracy, respectively.Training gains are 3.1 and 5.6 percentage points for Qwen3-4B and Qwen3-30B, respectively.
  • Model Configurations: Scientific Judge is evaluated across Qwen2.5 scales from 1.5B to 32B parameters and across Qwen2.5, Qwen3, and Llama model families.All variants are trained from their corresponding base models using GRPO on SciJudgeBench.

B.6 Release-Time Robustness

The evaluation tests whether RLCF remains effective beyond model release-time boundaries and whether Scientific Judge preserves general capabilities after preference training.

  • Release-time robustness: 904 pairs in the 2025 Temporal OOD set postdate both Qwen2.5-Instruct and Llama-3.1-8B-Instruct release dates, providing a conservative no-pretraining-exposure test.The models were released on 19 September 2024 and 23 July 2024, respectively.
  • Release-time robustness: Qwen3-4B improves from 68.6 to 80.4 (+11.9 percentage points) in the strictly post-release September–December window.August was excluded as the release-boundary month.
  • General capability preservation: Scientific Judge is evaluated on MMLU-Pro, GPQA, MATH, GSM8K, and SimpleQA to test whether specialized training degrades general capabilities.The benchmarks cover general knowledge, graduate-level science, mathematical reasoning, and factual accuracy.
  • General capability preservation: Qwen3-4B changes range from −0.5 to +0.6 percentage points across the five benchmarks, while Qwen3-30B-A3B changes by −0.22 points on average.For Qwen3-4B, MATH changes from 78.6% to 79.1% and GSM8K from 93.3% to 93.9%.

C.2 Evaluation Protocol

Scientific Thinker ideas are evaluated through randomized, majority-voted comparisons by three strong LLMs, with the voting procedure validated on SciJudgeBench.

  • Evaluation protocol: Each research-idea pair is judged by GPT-5.2-high, GLM-5, and Gemini 3 Pro at temperature 0.0, with majority voting selecting the winner.An idea wins when it receives at least two votes.
  • Evaluation protocol: The two ideas are randomly order-swapped with 50% probability before each evaluation to mitigate positional bias.This randomization is applied separately for each judge’s evaluation.
  • Metric validation: 84.4% accuracy on SciJudgeBench validates majority voting as a reasonable metric for assessing Scientific Thinker.The accuracy is reported for the voting method itself.

D Pairwise Comparison of Divergent Impact Series

The paper defines pairwise potential-impact ordering through long-run differences in cumulative expected citations, so comparisons remain meaningful even when individual impact series diverge.

  • Definition: Potential impact is defined through the limit behavior of cumulative expected citation differences between two papers.The ordering uses relative comparison rather than requiring finite absolute impact values.
  • Proposition: Even if I(p_a) = +∞ and I(p_b) = +∞, the ordering p_a ≻ p_b is well-defined whenever the relevant difference limit exists.The comparison can therefore remain defined when both individual cumulative series diverge.
  • Proof: A finite limit of the difference sequence establishes the comparison regardless of whether either individual cumulative impact series converges.Convergence of the difference does not require convergence of the two series separately.
  • Proof: If the difference diverges to +∞, paper p_a eventually and persistently dominates p_b in cumulative expected citations.This yields p_a ≻ p_b under the paper’s ordering definition.
  • Sufficient condition: Eventual non-negativity of expected citation differences is a sufficient condition for the difference sequence to converge in ℝ∪{+∞}.The paper describes this as a mild practical assumption because higher-impact papers typically maintain a persistent citation advantage after an initial period.
  • Implication: This pairwise ordering underlies JudgeCap and ThinkerCap, which rely on relative ordering rather than absolute impact values.The result justifies using pairwise comparisons in both capacity definitions.

E.1 In-Domain Example

The examples compare base-policy and Scientific Thinker ideas across RLVR exploration and financial reasoning, with evaluators favoring the trained proposals for broader or more fundamental approaches.

  • RLVR exploration: The RLVR base-policy idea uses distributional priming to inject controlled diversity into the model’s output distribution before training.It targets the limitation that RLVR amplifies the base model’s initial solution distribution.
  • RLVR exploration: Scientific Thinker proposes UG-VR, which uses model uncertainty to guide exploration toward low-probability but potentially correct solution paths.The proposal adds a confidence-aware exploration bonus while maintaining verifiability.
  • RLVR exploration: Scientific Thinker wins the RLVR comparison because evaluators describe UG-VR as more fundamental, reusable, and easier to adopt than the base policy’s static pre-training strategy.The conclusion characterizes the trained proposal as a dynamic, in-training algorithmic solution.
  • Financial reasoning: The financial base-policy idea proposes querying for missing information during reasoning when financial data are incomplete or ambiguous.It frames reasoning as active data acquisition rather than passive inference.
  • Financial reasoning: Scientific Thinker proposes MarketFlow, a framework that continuously adapts to streaming market events instead of relying only on historical-data fine-tuning.The proposal targets real-time market conditions as a practical boundary for financial reasoning.
  • Financial reasoning: Scientific Thinker wins the financial comparison because evaluators judge its online adaptation framework more broadly applicable and fundamental than the base model’s static solution.The comparison contrasts continuous evolution with a narrow response to data incompleteness.

E.2 Out-of-Domain Example

The out-of-domain examples show Scientific Thinker generating ideas judged more impactful than baseline ideas, while Scientific Judge transfers its judgement across future papers and evaluation settings.

  • Base Policy: The base policy proposes specialized agents collaborating through dynamic delegation and cross-domain verification in the CoRAG framework.Its design decomposes complex problems into domain-specific subtasks handled by lightweight agents.
  • Scientific Thinker: Scientific Thinker proposes a dynamic tool-adaptation framework judged more foundational and practically applicable than the base policy’s incremental multi-agent collaboration.The framework targets integrating new tools without retraining.
  • OOD Year: Scientific Judge correctly compares future-year papers despite their being outside the training period.The appendix explicitly presents 2025 comparisons as extrapolation beyond training.
  • OOD Year: In one 2025 comparison, the model prioritizes benchmark salience and broad community relevance over technical rigor in a niche subfield.The position-swapped prediction remains correct.
  • OOD Year: The model also infers citation potential from ecosystem adoption, infrastructure relevance, visibility, brand effects, and broad community attention.These cues support correct predictions in additional recent-paper comparisons.
  • OOD ICLR: Scientific Judge transfers citation-style judgement to ICLR acceptance signals, using venue fit, theoretical depth, and acceptance likelihood.The cited case reports a correct prediction for the ICLR comparison.

F.3 In-Domain Examples (Main Dataset)

In-domain examples show Scientific Judge identifying broadly reusable, foundational work across physics, mathematics, computer vision, quantum computing, and creative AI, with performance varying by model scale.

  • Mathematics: The model identifies foundational work whose downstream footprint spans multiple mathematical subareas rather than relying only on topical similarity.The pure-mathematics case explicitly frames downstream breadth as the decisive cue.
  • Physics: Scientific Judge recognizes durable methodology papers in specialized physics when broad reuse is the main citation driver.The N-subjettiness example contrasts a widely used jet-substructure method with a more limited observable.
  • 3D Computer Vision: A software-library citation heuristic favors broadly adopted infrastructure over a single-task method paper.The 3D computer-vision example contrasts Open3D with PU-Net.
  • Computer Vision: The 4B and 30B models both produce incorrect predictions in one computer-vision case, showing that learned judgement is not uniformly reliable.The appendix reports separate incorrect outcomes for both model sizes.
  • Others: The examples extend coverage beyond the core Computer Science and Physics categories to music generation and other creative-AI work.The music-generation case is described as successful when visibility and broad creative-AI relevance are strong.
  • Model Scale: Pure mathematics reveals a scale-dependent gain: the 4B model overweights foundationality, whereas the 30B model better tracks broader downstream usage.The appendix identifies this as its clearest scale-comparison example.
Loading 2603.14473v3…