PaperlayerView exact revision on arXiv

Source-linked AI summary

DiscoveryRSI: Recursively Learning to Discover Valuable Hypotheses

Jianyuan Zhong, Chenwei Wu, Zhaochen Yu, Xixian Jiang, Xuewei Li, Hao Xiao, Xilin Dai, Junyang Wang, Jiaru Zou, Zhiqing Cui, Boyuan Zheng, Jing Xiong, Zhen Zhang, Qiang Gao, Michael Morley, Luis Nakayama, Qiang Xu, Liyue Shen, Tao Gui, Leo Anthony Celi, James Zou, Zhenfei Yin, Yingcheng Wu, Ling Yang

arXiv:2610.23839v1cs.CL

TL;DR

Research agents need to turn evolving hypothesis development and investigation into learning that improves later research. DiscoveryRSI preserves research histories and uses them for recursive model and workflow improvement, with reported gains in hypothesis quality and research-path development.

  • Problem

    Research agents face the challenge of using experience from hypothesis development and investigation to improve future research, while linking assessments to the contributions behind them.

    Source: §1 · pp. 2, 4

  • Method

    DiscoveryRSI develops hypothesis trees, compiles selected investigations into executable Harbor tasks, and recursively learns through contribution-based credit, harness replay, and rediscovery.

    Source: §1 · p. 4

  • Results

    Hypothesis quality rises from 4.86 to 5.39, and DiscoveryRSI-27B achieves 6.3% hypothesis correspondence with post-cutoff research versus 2.5% for GPT-6 and 2.3% for Claude Opus 5.

    Source: §1 · p. 5

  • Takeaways & Limitations

    The present work reaches concrete hypotheses and compiled investigation plans, while execution and learning from experimental outcomes remain next steps.

    Source: §9 · p. 19

    The work reaches concrete hypotheses and compiled plans but still requires execution, wet-lab validation, outcome-based research decisions, and evaluation in additional scientific domains.

    Source: §9 · p. 19

Abstract

from arXiv · show

Scientific discovery requires developing worthwhile hypotheses into practical investigations, yet learning from this evolving research process remains challenging. We introduce an operational definition of valuable and testable research hypotheses: evidence-grounded claims about consequential uncertainties, coupled with feasible investigations, explicit decision criteria, and alternative explanations. Our DiscoveryRSI framework develops hypothesis trees through multi-agent evolutionary search, compiles selected investigations into executable Harbor tasks, and recursively learns through three mechanisms: hypothesis credit optimization attributes scientific assessments to documented research contributions; harness replay identifies productive strategies and recurring failures to refine research workflows; and rediscovery applies accepted improvements to new investigations, generating further experience for subsequent learning. We also introduce DFM-Bench, which provides comprehensive assessment of hypothesis quality and cumulative research progress at the levels of hypotheses, revisions, development paths and research runs. Across four life-science domains, automated evaluation shows that hypothesis quality rises from 4.86 for the base model to 5.39 after two rounds of self-improvement, with the large gains in falsifiability and data resolvability. We further validate hypothesis quality through expert review, comparing our DiscoveryRSI-27B with other research agents. Preliminary human assessments show that DiscoveryRSI-27B scores higher than GPT-6 Luna on hypothesis importance and whether claims are plausible and evidence-bounded. DiscoveryRSI-27B achieved a 6.3% rate of hypothesis correspondence with research published after the model cutoff, largely outperform GPT-6 with 2.5% and Claude Opus 5 with 2.3%, while delivering greater human-rated improvement along research paths.

1. Introduction

Research agents must turn evolving hypothesis development and investigation into learning that improves subsequent research. DiscoveryRSI preserves research histories, uses them to refine models and workflows, and evaluates hypothesis quality and cumulative research progress.

1. Introduction

  • Research agents must connect hypothesis development and experimental execution with learning from experience, while preserving histories that link assessments to the actions behind them.Final assessments alone may obscure which contributor supplied an explanation, useful control, or evidence-driven revision to an investigation plan.Source: §1 · pp. 2, 4
  • Earlier systems develop hypotheses, connect generation with execution, or turn scientific interactions into learning tasks; DiscoveryRSI focuses on learning from hypotheses agents actively develop.It preserves how proposals evolve, links quality assessments to contributions, and uses that experience to improve subsequent investigations.Source: §1 · pp. 2, 4
  • DiscoveryRSI develops testable hypothesis trees through multi-agent evolutionary search, compiles selected investigations into Harbor tasks, and uses credit optimization, harness replay, and rediscovery for recursive improvement.Credit optimization links assessments to documented contributions; harness replay identifies productive strategies and recurring failures; rediscovery applies accepted updates to fresh investigations.Source: §1 · p. 4
  • DFM-Bench assesses hypothesis quality and cumulative progress across hypotheses, revisions, development paths, and research runs, alongside expert-supervised review and controlled agent comparisons.The paper reports clear gains over self-improvement rounds and changes in agent behavior patterns.Source: §1 · pp. 4–5
  • Hypothesis quality rises from 4.86 to 5.39; DiscoveryRSI-27B achieves 6.3% hypothesis correspondence with post-cutoff research, versus 2.5% for GPT-6 and 2.3% for Claude Opus 5.Expert-supervised assessment also reports an average gain per revision of +0.07, compared with +0.03 for GPT-5.6 Sol and −0.01 for GPT-6 Luna.Source: §1 · p. 5

2. Valuable and Testable Research Hypotheses

The paper defines valuable and testable research hypotheses as revisable objects that connect evidence-grounded claims to feasible investigations and interpretation criteria. It emphasizes forming questions from observations and retaining the plans and evidence needed to assess them.

2.1. An Operational Definition

  • A valuable, testable research hypothesis combines an evidence-grounded claim, feasible investigation plan, decision criteria, and relevant alternative explanations.Value concerns consequential uncertainty relative to the evidence base; testability requires observable outcomes that can support, weaken, or bound the claim under declared constraints.Source: §2.1 · p. 5

2.2. From Observations to Research Questions

  • Research questions are formed by tying observations and available evidence to a defined system or population, observable quantities, and interpretable outcomes.Descriptive analyses of patterns, variation, and measurement coverage help identify questions worth pursuing and the evidence available to investigate them.Source: §2.2 · p. 6

2.3. A Hypothesis with an Investigation Plan

  • The MIMIC-IV example tests whether repeated creatinine–urine output mismatch adds information about adverse outcomes after ICU discharge using held-out comparisons and explicit interpretation criteria.Recorded versions link proposals to predecessors and supporting evidence, making changes in questions, plans, and interpretations traceable.Source: §2.3 · p. 6

3. DiscoveryRSI

DiscoveryRSI connects hypothesis development and executable investigations to recursive learning from research histories. Its linked records support credit assignment, workflow refinement, and fresh investigations using accepted updates.

3. DiscoveryRSI

  • DiscoveryRSI links hypothesis development, executable investigations, and retained research records to support learning across investigations.The complete protocol moves from within-run development to accepted updates and fresh data collection.Source: §3 · p. 6

3.1. Research Trees and Hypothesis Development

  • Research trees store versioned hypotheses with plans, decision criteria, alternatives, and evidence, while ancestry links record revisions and combinations.Agent actions remain distinct from the scientific development sequence; selected directions evolve through hypothesis development, repair, and crossover.Source: §3.1 · pp. 6–8

3.2. From Hypotheses to Executable Investigations

  • Selected computational plans become Harbor tasks with inputs, environments, required outputs, and verification checks, while unavailable data and physical experiments remain dependencies.Execution checks test the task contract; separate scientific assessment determines what the investigation can establish.Source: §3.2 · p. 8

3.3. Hypothesis Credit Optimization

  • Hypothesis credit optimization links scientific assessments to documented contributor histories, then trains on supported assistant actions rather than every recorded message.Credit is bounded and unsupported shares remain unallocated; tool observations and known failed actions may provide context but receive no direct loss.Source: §3.3 · pp. 8–9

3.4. Harness Replay

  • Harness replay uses diagnostic runs and recorded histories to identify recurring failures, propose targeted edits, and screen changes under matched resource limits.Matched development evaluation determines acceptance using scientific quality, exploration breadth, reliability, and cost; the incumbent remains eligible.Source: §3.4 · p. 9

3.5. Rediscovery

  • Rediscovery applies accepted model and harness updates to fresh investigations, collecting new hypotheses, evidence, assessments, and contribution histories.Updates may be accepted separately or jointly, so the model, harness, both, or neither may change.Source: §3.5 · pp. 9–10

4. DFM-Bench: Evaluating Scientific Quality and Research Progress

DFM-Bench assesses scientific inquiry at multiple connected levels, from individual hypotheses to distinct contributions. It makes revisions, development paths, and whole research runs explicit assessment objects alongside existing benchmark constructs.

4. DFM-Bench: Evaluating Scientific Quality and Research Progress

  • DFM-Bench connects assessments of hypothesis merit, revision value, development-path progress, research-run quality, and scientific contribution.Each question requires evidence appropriate to its level.Source: §4 · p. 10

4.1. Five Levels of Scientific Assessment

  • Hypothesis quality assesses whether an individual proposal is worthwhile and testable across importance, novelty, grounding, falsifiability, data resolvability, feasibility, alternatives, and actionability.An aggregate score is produced when required assessments are complete.Source: §4.1 · pp. 10–11
  • Revision-value assessment compares a hypothesis with its actual predecessor and distinguishes substantive progress, necessary repair, repetition, and insufficient evidence.Score changes are recorded separately from these judgments.Source: §4.1 · p. 11
  • Development-path assessment follows ordered ancestry to judge evidence acquisition, coherent revision, scientific continuity, and decisions to continue or stop.Agent action histories support these judgments, while the path records how hypotheses changed.Source: §4.1 · p. 11
  • Research-run assessment examines the full research graph, exploration, synthesis, selection and stopping decisions, and budget allocation rather than averaging path scores.Opportunity and resource records inform the assessment.Source: §4.1 · p. 11
  • Scientific-contribution assessment groups related proposals to identify additions beyond starting evidence and distinguishes proposed ideas from contributions supported by completed investigations and independent checks.Possible additions include a new distinction, informative negative result, supported finding, or reduced uncertainty.Source: §4.1 · p. 11

4.2. Coverage Relative to Existing Benchmarks

  • DFM-Bench makes hypotheses, revisions, development paths, research runs, and contribution families explicit assessment objects beyond complementary benchmark coverage.The comparison concerns evaluation constructs, not the capabilities of agents running on those benchmarks.Source: §4.2 · p. 11

5. Expert-Supervised Assessment of Scientific Hypotheses

The expert-supervised assessment examines whether generated hypotheses address worthwhile questions and support interpretable investigations. Reviewers assess anonymized proposals using value judgments, anchored criteria, and design-flaw verdicts.

5. Expert-Supervised Assessment of Scientific Hypotheses

  • The review cohort included 292 proposals from six dataset groups spanning four life-science domains, assessed by 27 medically or bioscience-trained participants under three senior supervisors.Proposals came from a fixed hypothesis-atlas snapshot and were presented in randomized order.Source: §5 · p. 11
  • Reviewers received proposal packets with questions, plans, data requirements, alternatives, and citations, while producer provenance and automated grades were withheld.Reviewers assessed each proposal on its own merits without consulting its revision history.Source: §5 · p. 12
  • Reviewers recorded an overall value judgment and recommendation, rated six anchored criteria, and separately judged whether a consequential design flaw was absent, fixable, or fatal.Reviewers could select “Can’t judge” when evidence or expertise was insufficient; flaw details and comments were optional.Source: §5 · p. 12, Figure 4 · p. 12

6. Experimental Setup

The experimental setup evaluates DiscoveryRSI across life-science domains and adds a battery setting to test transfer beyond life science. Evaluation combines automated quality judgments, behavioral analysis, human review, and literature-based assessment.

6. Experimental Setup

  • DiscoveryRSI is evaluated across clinical and population health, target prioritization, disease mechanisms, and population multi-omics using clinical, population, and molecular datasets.Inputs include HCC, MIMIC-IV, eICU, UK Biobank, GEO, and ArrayExpress data.Source: §6 · p. 12
  • The added battery setting covers four domains and uses battery-specific checks for operating conditions, sensing, physical plausibility, and competing explanations.Investigations separate measured quantities from fitted degradation labels and specify cell, condition, and protocol splits.Source: §6 · p. 13
  • Hypothesis-credit training fine-tunes DiscoveryRSI-27B on 2,397 trajectories selected from 15,792 collected research trajectories.The sequences contain 291.12 million tokens, with 6.18 million (2.12%) receiving direct supervision; training runs for one epoch.Source: §6 · p. 13
  • Evaluation covers automated hypothesis quality across 241 questions, six behavioral campaigns, human review of 292 proposals, and literature-based assessment with model-specific cutoffs.Human assessment hides model identities and automated scores; behavioral analysis uses 20 episodes per evaluated model variant.Source: §6 · p. 13

7. Results

Across the reported evaluations, hypothesis quality rises over self-improvement rounds, with gains concentrated in testable investigation design. Research behavior shifts toward sustained refinement, while human review and literature comparison provide additional assessments of revision progress and hypothesis inspiration.

7. Results

  • 5.61: battery-domain mean hypothesis quality across 165 complete scores, with substantive progress on 58.47% of transitions and a mean trajectory score of 81.52.The evaluation covers 200 episodes and 351 generated hypotheses; the research round score is 71.25.Source: §7 · p. 13

7.1. RQ1: Does learning from research histories improve hypothesis quality?

  • Falsifiability rises from 4.00 to 6.20 and data resolvability from 3.73 to 4.99, while importance falls from 6.48 to 6.04 and novelty from 4.87 to 4.58.The reported improvements center on specifying testable investigations rather than increasing perceived significance or originality.Source: §7.1 · p. 14

7.2. RQ2: How does DiscoveryRSI change research behavior?

  • Evolve and repair operations rise from 77.5% to 91.7% of episodes, while restarts fall from 21.7% to 5.0%; development-path gains are largest in evidential and inferential discipline (+1.25).Changes in cumulative progress, stopping decisions, and budget efficiency remain below 0.1 points, while disciplinary connections decline by 0.28; action frequencies alone do not establish revision quality.Source: §7.2 · pp. 14, 16

7.1. RQ1: Does learning from research histories improve hypothesis quality?

  • 4.86 to 5.39: mean automated hypothesis quality rises from the base model to round two, with gains in all four domains.The first-round mean is 5.18; the additional gain decreases from 0.32 in round one to 0.21 in round two.Source: §7.1 · p. 13, Figure 6 · p. 15

7.3. RQ3: Do human reviewers recognize improvements in the hypotheses?

  • 41% of DiscoveryRSI-27B revisions improve human scores, 47% are unchanged, and 12% worsen; mean change per revision is +0.07, versus +0.03 for GPT-5.6 Sol and −0.01 for GPT-6 Luna.These assess progress within research trees, complementing comparisons across rounds.Source: Figure 8 · p. 16, §7.3 · p. 16
  • DiscoveryRSI-27B scores higher than GPT-6 Luna on importance (3.10 versus 2.94) and plausible, evidence-bounded claims (3.37 versus 3.16), but scores for data fit, alternative explanations, and informative results favor GPT-6 Luna.In a MIMIC-IV path, six-criterion mean ratings rise from 2.83 to 3.50 as revisions clarify timing, data fields, and acute kidney injury thresholds; two depth-four siblings still lack specified thresholds.Source: §7.3 · pp. 16–17, Figure 9 · p. 17

7.4. RQ4: Do the hypotheses inspire subsequent research?

  • 6.3% of DiscoveryRSI hypothesis versions meet the literature-inspiration criterion, compared with 5.4% for DeepSeek, 2.5% for GPT, and 2.3% for Claude.Across DiscoveryRSI domains, rates range from 16.9% in therapeutic targets to 2.3% in clinical and population health; group sizes, domain composition, cutoffs, and retrieval coverage differ, and prospective tracking is needed to establish adoption.Source: §7.4 · p. 17, Figure 10 · p. 18
  • Only 15 of Claude Opus 5’s 131 proposals pass the retrieval gate, versus 355 of 1,176 for DiscoveryRSI-27B; among retrieved proposals, positive-judgment fractions are similar at 20.0% and 20.8%.A title-query diagnostic retrieves 95 Claude proposals, but does not establish additional inspiration matches.Source: §7.4 · p. 18

8. Related Work

Related work spans research agents, scientific evaluation, learning from assessed experience, and persistent workflow improvement. DiscoveryRSI connects these directions through hypothesis-centered records of claims, evidence, and investigation decisions.

Discovery agents and evaluation

  • Research agents develop hypotheses through multi-agent refinement [8], connect ideation to experimental search [33], or integrate biomedical tools and data.Related evaluation benchmarks cover ideation, data-driven discovery, executable investigations, and research-process assessment.Source: §8 · pp. 18–19

Learning from evaluated research experience

  • Reward-guided methods learn from rated samples, feedback-filtered generations, weighted objectives, or scientific exploration trajectories [30] [24] [37].DiscoveryRSI targets the attribution problem of linking proposal assessments to documented contributions in the histories that produced them.Source: §8 · p. 19

Persistent and recursive agent improvement

  • Persistent-improvement work retains experience in memories, prompts, playbooks, programs, or selected executions [1] [29] [41] [39] [40] [16].DiscoveryRSI applies these directions to hypothesis development, using shared records for contribution-based learning, harness replay, and rediscovery.Source: §8 · p. 19

9. Conclusion

The conclusion presents DiscoveryRSI as a way to connect hypothesis development with learning from documented research contributions. It identifies proposal selection, wet-lab validation, and broader domain coverage as next steps.

9. Conclusion

  • DiscoveryRSI connects hypothesis development with learning from documented research contributions.The present work reaches concrete hypotheses and compiled investigation plans; determining which proposals merit investigation remains a next step.Source: §9 · p. 19
  • Closing the discovery loop further requires wet-lab validation and methods that learn from supportive, negative, and inconclusive outcomes.Source: §9 · p. 19
  • Extending hypothesis discovery and DFM-Bench to additional scientific domains is proposed to help establish how these capabilities generalize.Source: §9 · p. 19

DiscoveryRSI

DiscoveryRSI links hypothesis development to executable investigations and uses documented research records to support recursive learning. Its appendices detail the roles, provenance, assessment, and allocation procedures behind that workflow.

A.1. Research Objects and Agent Roles

  • Hypotheses are recorded as proposals and revisions in a generally directed acyclic research graph, with distinct agent roles for selecting, generating, evolving, critiquing, assessing, and compiling work.Evolution and repair retain a direction, crossover combines directions, and restart opens a new question.Source: §A.1 · p. 24

A.2. Record Types and Provenance

  • Persistent records link hypotheses to evidence, data, contributions, and execution; provenance links record material use or production, while scientific assessment evaluates its support for claims.Files retain material and explicit references retain provenance and relationships.Source: §A.2 · pp. 24–25, Table 2 · p. 24

A.3. From Contributions to Executable Investigations

  • Contribution attribution links assessed hypotheses to documented producing, supporting, and coordinating trajectories, while unresolved associations receive no allocated share.Shared episode membership, ancestry, or worker participation alone does not establish a credit edge.Source: §A.3 · p. 25, §B.3 · p. 28
  • Selected computational plans become Harbor tasks with inputs, environments, outputs, and verification checks, while unavailable measurements and wet-lab experiments remain explicit dependencies.Compilation assesses the computational package; completed execution and scientific assessment establish what evidence the investigation supplies.Source: §A.3 · p. 25

A.4. Harness Replay and Rediscovery

  • Harness replay diagnoses failures and screens repairs using recorded observations; alternatives require recorded continuations or new execution from a compatible restored state.Acceptance comparisons hold checkpoint, inputs, data access, and resource limits fixed before candidate changes are deployed on fresh investigations.Source: §A.4 · p. 25

B.1. Pairwise Preference Assessment

  • Blinded pairwise Critique outcomes inform relative hypothesis-preference estimates, while separate ancestor-aware Progress judgments assess transitions against predecessors.The stored preference marginals are approximate and do not fully represent reviewer bias, scientific uncertainty, or dependence between repeated critiques.Source: §B.1 · pp. 26–27

B.2. Hypothesis Weights

  • Hypothesis weights combine a conservative, tie-aware within-stratum rank with reliability factors and a bounded outcome transformation; missing assessments remain unknown, not negative labels.The Progress bonus supplements the outcome weight without changing the preference posterior.Source: §B.2 · pp. 27–28

B.3. Contributor Shares and Provenance; B.4. Scientific-Family Credit Limits; B.5. Objective Balancing

  • Contributor shares follow documented contribution provenance, while local and global scientific-family budgets limit credit for repeated versions of the same question.The implemented example uses fixed responsibility fractions, not a Bayesian estimate of actors’ marginal contribution; objective balancing separately assigns domain and group influence.Source: §B.3 · p. 28, §B.4 · p. 29, §B.5 · p. 29

Appendix C. Training Data and Implementation

Appendix C describes how DiscoveryRSI training trajectories are selected, weighted, serialized, and windowed. Its windowing equivalence result depends on retaining the original conditioning context, which the implemented long-trajectory export does not generally preserve.

C.1. Trajectory Collection and Selection

  • From 15,792 collected research trajectories, 2,397 agent trajectories remain in the training export after review, attribution, and tokenization.The export excludes 35 trajectories with unsupported mid-conversation developer messages.Source: §C.1 · p. 31, Table 4 · p. 31

C.2. Supervision and Optimization

  • The offline weighted-supervision objective applies loss to supported assistant responses and tool-call arguments, assigning zero direct loss to observations, failed actions, repeated context, and unsupported actions.Domain balancing assigns 90% of objective weight to life-science trajectories and 10% to battery-research trajectories.Source: §C.2 · p. 31

C.3. Training Configuration

  • Training uses full-parameter fine-tuning with BF16 precision, Adam, a 5 × 10−6 learning rate, global batch size 16, and one epoch over 150 optimizer steps.The distributed configuration uses 64 GPUs.Source: §C.3 · p. 32

C.4. Serialization and Target Selection

  • Serialization preserves source records and masks eligible assistant outputs for direct loss, while known failed actions, repeated context, and unavailable reasoning placeholders receive zero direct loss.Preparation checks validate the training representation, not the scientific correctness of its underlying hypotheses.Source: §C.4 · p. 32

C.5. Windowing: Preserved Targets and Conditional Equivalence

  • The windowing proposition gives identical objectives and gradients only when targets retain their original conditioning context, windows share a fixed coefficient, and normalization uses the full target count.Repeated context receives zero loss.Source: §C.5 · p. 33
  • The implemented 256K export preserves target counts and fixed normalization but can omit earlier context, so it does not generally preserve full-history likelihoods or gradients.Independently renormalizing each window would also violate the proposition’s assumptions.Source: §C.5 · p. 33

Appendix D. DFM-Bench Evaluation Protocol

DFM-Bench defines evidence-based assessments for hypotheses and increasingly cumulative research objects, while keeping scientific quality distinct from delivery maturity. Its protocols make missing evidence, applicability, and evaluator calibration explicit across hypothesis development, human review, and literature-inspiration comparisons.

D.1. Hypothesis Quality and Automated Review

  • DFM-Bench assesses hypotheses across eight weighted dimensions, while its review panel’s separate prompts to one model family provide correlated AI perspectives rather than independent expert opinions.Complete quality scores require three reviews, majority eligibility, and medians for all eight dimensions; unknown evidence remains unknown.Source: §D.1 · pp. 33–34, Appendix D · p. 33

D.2. Revisions, Development Paths, and Research Runs

  • Research-progress assessments distinguish revisions, selected development paths, and whole runs; shared ancestors correlate path measurements, and research-run scores are not averages of path scores.Path coverage counts selected longest-depth paths to terminal nodes rather than every possible route through the graph.Source: §D.2 · p. 34

D.3. Scientific Contribution, Delivery, and Calibration

  • Scientific-contribution families record increments beyond starting evidence without implying confirmation, while delivery maturity remains separate and human calibration is still pending.The framework distinguishes current knowledge progress from future capability growth; retrospective maxima over final-snapshot scores cannot demonstrate learning.Source: §D.3 · p. 35

D.4. Development-Path and Research-Run Rubrics

  • The cited rubric assigns 20% each to cumulative scientific progress (L3) and evidential and inferential discipline (L4); dimension scores use 0–4 evidence anchors qualified by domain-specific requirements.Source: §D.4 · p. 35

D.5. Assessment Records and Missing Evidence

  • Assessments retain their object, source snapshot, protocol, evidence, scores, rationale, and review status; unknown values are not zero, and incomplete records receive no complete composite.Resource-allocation judgments require recorded budgets and opportunity information.Source: §D.5 · p. 36

D.6. Benchmark Coverage Definitions

  • DFM-Bench distinguishes hypothesis merit, contribution maturity, evidence use, revision, path, and whole-run allocation; benchmark comparisons describe construct coverage, not agent capability.Its checks indicate implemented rubric coverage conditional on sufficient records, not independent expert validation or empirical superiority over other instruments.Source: §D.6 · pp. 36–37

Appendix E. Human Assessment Protocol

  • Human review separates five-point overall research value from four-point criterion ratings and design-flaw verdicts, using 27 biomedical or clinical-background reviewers and 292 proposals across four life-science domains.Reviewers saw randomized, anonymized proposal packets without model identity, execution provenance, or automated grades.Source: §E.2 · p. 37, §E.1 · p. 37, Appendix E · p. 37

Appendix F. Literature-Inspiration Evaluation

  • The literature-inspiration comparison uses operational cutoffs and retrieval plus automated judgments; DiscoveryRSI-27B’s rate is 74/1,176, while group sizes and reused judgments differ and results are descriptive.The corpus was shared across groups within the stated publication window, but operational cutoffs do not identify internal knowledge cutoffs; missing positive labels can reflect retrieval or audit coverage.Source: §F.1 · p. 38, §F.2 · p. 38, §F.3 · p. 39

F.4. Claude Opus 5: Retrieval Sensitivity

Claude Opus 5’s lower measured inspiration rate coincides with lower retrieval coverage, while positive-judgment fractions among retrieved proposals are similar. The title-query diagnostic increases retrieval, indicating representation sensitivity rather than additional positive semantic matches.

F.4. Claude Opus 5: Retrieval Sensitivity

  • 15/131 Claude proposals passed retrieval (11.5%) versus 355/1,176 DiscoveryRSI proposals (30.2%), while retrieved-proposal positive fractions were similar at 20.0% and 20.8%.Using proposal titles as queries increased Claude’s retrieval count from 15 to 95, but did not establish additional positive semantic matches.Source: §F.4 · p. 39
  • The audit measures literature correspondence, not researcher exposure, adoption, study initiation, or causal influence.Prospective evaluation would freeze hypotheses before researcher review and follow subsequent plans, experiments, and publications.Source: §F.4 · p. 39

Related papers

Loading 2610.23839v1…