Accelerating Scientific Research with Gemini in the Real-World

Materials science

• Co-Scientist designed a non-hazardous CVD precursor route yielding Ti3C2Tx-like layered structures, and achieved single-attempt growth of monolayer MoS2, MoSe2, and WS2.The Ti3C2Tx atomic structure remains unconfirmed, while Gemini 3 Deep Think enabled recipe adaptation to laboratory constraints in minutes.Source: §1 · p. 2, Figure 1 · p. 3

Biology

• Co-Scientist predicted engineered E. coli swarming phenotypes across IPTG gradients from sparse imaging data, quantitatively matching unpublished wet-lab morphological measurements.Domain experts refined the task framing and performed the wet-lab assays, suggesting potential reductions in combinatorial screening cycles.Source: §1 · p. 2, Figure 1 · p. 3

Computer science

• Given only a research directive, Co-Scientist autonomously discovered an inference-time scaling architecture that outperformed six frontier models on HealthBench Hard and Professional.The architecture also achieved a significant, though modest, reduction in potential clinical harm under blinded physician evaluation.Source: §1 · p. 2, Figure 1 · p. 3

Scientific integrity

• A double-blind study of 30 domain experts across 450 independent reviews found that log-based verification and safety mechanisms reduced hallucination and plagiarism versus unconstrained baselines.The study evaluated end-to-end generated papers in computational science.Source: Figure 1 · p. 3, §1 · p. 4

• Across materials science, biology, and computer science, execution-grounded Co-Scientist bridges in silico exploration and physical reality through human–AI collaboration and autonomous discovery.The system adapts collaboration to domain constraints while coupling iterative reasoning with laboratory feedback.Source: §1 · pp. 2, 4, Figure 1 · p. 3

2. Methods

• Co-Scientist evolves hypotheses, experimental programs, and manuscripts through sequential ideation, experimentation, and paper-writing stages.Ideation combines literature grounding, Bayesian pairwise tournaments, UCB exploration, crossover, and mutation; experimentation iteratively generates and executes programs; paper writing scaffolds and refines structured manuscripts.Source: §2 · pp. 4–5

2.1. Reducing hallucination and plagiarism

• The reliability design jointly optimizes reviewer quality with plagiarism and hallucination penalties, while hallucination clipping verifies quantitative claims against raw execution logs.The default weights are λ_review=1.0, λ_plag=0.5, and λ_hall=1.0; the hallucination term is conditioned on experimental source code and execution logs.Source: §2.1 · p. 5, §2.1.1 · pp. 5–6

2.1.1. Reliability via joint-optimization

• For ideation, literature-grounded hypotheses undergo reflection and pairwise Bayesian ranking with UCB exploration, while crossover and mutation prioritize uncertain, potentially novel directions.The reflection agent evaluates novelty, plausibility, and testability and filters unauthorized equipment claims; UCB prioritizes hypotheses with higher uncertainty before scores converge.Source: §2.1.1 · p. 6

2.1.2. Further claim verification and factual alignment

• The claim-verification module parses statistical assertions and performance metrics, rewrites discrepancies using verified log data, and stops paper writing when no valid execution logs or results exist.Verbose logging is encouraged so execution records contain sufficient detail for factual alignment before manuscript finalization.Source: §2.1.2 · pp. 6–7

2.2. Reducing harmful research

• A two-layer safety architecture screens user research directions for dual-use concerns and continuously evaluates ideas and plans, returning corrective feedback for disapproved designs.The initial ethics screen can refuse restricted objectives, while ongoing evaluator feedback iteratively steers the agent toward non-harmful research without human intervention.Source: §2.2 · p. 7

3.1.2. Precursor discovery for bottom-up synthesis of 2D titanium carbide

• The C2Cl6-based protocol produced a layered crystal with Ti3C2T_x-consistent structural and chemical signatures, including ∼1.13 nm interlayer spacing and 2.51 Å in-plane spacing.SEM, EDS, and TEM supported the assignment, but oxidation, low yield, and absent cross-sectional atomic STEM prevented definitive phase confirmation.Source: §3.1.2 · pp. 10–11, Figure 2 · p. 12

3.1.3. One-take synthesis of 2D semiconductors

• Instrument-specific protocols produced high-quality monolayer MoS2 with triangular flakes exceeding 50 μm, alongside distinct irregular flakes and continuous films exceeding 80 μm × 80 μm.The MoS2 protocol was generated from hardware constraints without prior optimization history, and Raman peak separation of ∼21 cm−1 confirmed monolayer thickness.Source: §3.1.3 · pp. 11, 13, Figure 3 · p. 14
• Rapid Gemini 3 Deep Think inference generated machine-executable recipes in minutes and achieved first-attempt MoS2, MoSe2, and WS2 growth in approximately one hour, with smaller and less regular domains.Extensive evolutionary ideation produced higher-quality morphology but required approximately one day of test-time compute; MoSe2 and WS2 succeeded despite no prior laboratory synthesis experience.Source: §3.1.3 · p. 13, §3.1.4 · p. 15

3.1. Discussion

• Co-Scientist enabled bottom-up growth of an emergent Ti3C2T_x-like 2D phase using safer C2Cl6 and first-attempt monolayer synthesis of MoS2, MoSe2, and WS2.The materials studies combined evolutionary ideation with human oversight for precursor discovery and rapid Gemini 3 Deep Think inference for direct lab integration.Source: §3 · p. 7, §3.1.1 · p. 9, §3.1.4 · p. 13

3.1.4. Discussion

• After sealing and cleaning improvements, reproducibility increased from 11.5% (3 of 26 experiments) to 68.0% (17 out of 25 total experiments).The initial failures were traced to oxygen leaks, while further recipe optimization and protection from post-growth air exposure remain necessary.Source: §3.1.4 · pp. 13, 15

3.2. Predicting engineered E. coli swarming behavior

• For engineered E. coli, Co-Scientist was tasked with zero-shot interpolation of swarm morphologies across held-out IPTG concentrations from sparse endpoint images.It autonomously implemented a vision-language pipeline using neighboring-condition interpolation and Best-of-N rejection sampling with N = 16, while humans iteratively refined the task specification.Source: §3.2 · p. 16
• Co-Scientist predicted engineered E. coli swarming phenotypes across IPTG conditions, matching ground-truth trends for three of four morphological metrics while diverging only for pLac-rpoS circularity.Mean radius, eccentricity, and circumferential intensity CV were statistically consistent with experiments, with p=0.593, 0.451, and 0.712; circularity diverged at p=0.002.Source: §3.2 · p. 18, §3.2.1 · p. 18, Figure 5 · p. 19

3.2.1. Discussion

• The E. coli demonstration supports zero-shot phenotypic interpolation constrained by visual evidence rather than confabulation, but it tested interpolation along a known IPTG gradient rather than extrapolation.The approach may reduce physical sampling requirements, although extension to new genetic circuits and growth conditions remains necessary and it does not model causal mechanisms.Source: §3.2.1 · pp. 18, 20

3.3.1. Autonomous discovery of inference-time scaling architectures under constraints

• Co-Scientist discovered Agent_H, an eight-phase inference-time scaling pipeline combining triage, candidate generation, tournament selection, clinical auditing, verification, citation checks, and length optimization.Agent_H generates 28–48 candidates and uses approximately 40–80 LLM calls per query, depending on complexity.Source: §3.3.1 · p. 21, Figure 6 · p. 22
• Agent_H led all models on length-adjusted HealthBench Hard and Professional scores under the GPT-5.4 judge, while also leading Professional after length adjustment under Gemini 3.5 Flash.Under Gemini 3.5 Flash, Agent_H scored 0.377 on Hard and 0.643 on Professional after length adjustment; under GPT-5.4, it scored 0.292 and 0.619, respectively.Source: Table 2 · p. 23, §3.3.1 · p. 24
• Agent_H reduced physician-rated likelihood of harm versus Gemini 3.1 Pro, but showed no significant differences on the other eight clinical dimensions.The blinded evaluation covered 106 questions, with p=0.0486 after false discovery rate correction; autorater alignment with clinicians was low.Source: §3.3.1 · p. 24, Figure 7 · p. 25

3.3.2. Discussion

• The medical gains generalized from synthetic consumer-facing training queries to held-out clinician-facing HealthBench Professional prompts without evidence of benchmark data leakage.The architecture was developed without access to evaluation queries, cases, or rubrics, and decontamination analysis found no exact matches with comparable similarity profiles.Source: §3.3.1 · pp. 21, 23, §3.3.2 · p. 24
• Automated benchmark gains should be interpreted cautiously because physician evaluation found no broad quality advantage, reflecting low alignment between rubric-based autoraters and clinical preferences.Automated judges were internally consistent, with Spearman ρ=0.869, but benchmark rubrics may not capture valid practice variation or broader clinical utility.Source: §3.3.2 · p. 26

3.4.1. Study design: topic selection and paper generation

• The end-to-end paper study evaluates autonomous research reliability across 50 AI topics and 150 manuscripts generated under full-reliability, ablated, and Agent Laboratory baseline conditions.The full system included hallucination and plagiarism penalties, deterministic log verification, and ethical oversight; the ablated system removed the penalties and clipping module.Source: §3.4 · p. 26, §3.4.1 · p. 27
• The study gave the system only natural-language research topics, requiring autonomous resource acquisition and a three-stage ideation, experimentation, and paper-writing pipeline evaluated through blind expert reviews.Each run produced a research idea, experiment plan, Python code, execution logs, and compiled PDF manuscript; matched topics separated reliability-module effects from task difficulty and model capability.Source: §3.4.1 · pp. 27–28

3.4.2. Co-Scientist suppresses fabricated results

• Co-Scientist reduced severe result hallucinations to 4% versus 46% for ablation and 90% for baseline, eliminating extreme fabrication and lowering mean error severity to 0.78 versus 7.16 for baseline.The reliable system recorded zero complete-fabrication instances, while ablated and baseline systems recorded rates of 40% and 44%.Source: §3.4.2 · p. 28, Figure 8 · p. 29

3.4.3. Co-Scientist suppresses plagiarized findings

• Severe plagiarism fell to 16% versus 50% for ablation and 60% for baseline, while correct attribution in severe derivative cases reached 39.4% versus 15.9% for baseline.Co-Scientist produced 109 of 150 reviews classified as novel and only one direct-copying instance, whereas baseline manuscripts systematically recombined existing ideas.Source: §3.4.3 · pp. 28, 30, Figure 8 · p. 29

3.4.4. Ethical oversight prevents the generation of harmful research

• Ethical oversight yielded 96.3% safe ideas and 96.7% safe plans, compared with 86.7% and 72.3% after ablation, while refusing 98.7% of harmful directives and incorrectly refusing 3.1% of non-harmful ones.Safety did not reduce idea quality: expert ratings were 3.25 with oversight versus 3.26 for ablation (p=0.82).Source: §3.4.4 · p. 30

3.4.5. Discussion

• Severe result hallucinations fell to 4% versus 90% for Agent Laboratory, extreme data fabrication was 0% versus 44%, methodological divergence was 24% versus 100%, plagiarism was 16% versus 60%, and harmful directives were refused in 98.7% of cases.These findings came from 150 manuscripts evaluated by 30 domain experts across 450 blind reviews.Source: §3.4.5 · pp. 30–31
• The system autonomously selected diverse computational methods, but residual failures include selective reporting, code–mathematics divergences, and disguised mock functions, while 2× NVIDIA A100 GPUs limit experiments to lightweight models.Further progress requires semantic code inspection, complete reporting audits, and expanded computational resources.Source: §3.4.5 · p. 31

Automating narrow research tasks

• LLMs increasingly automate modular research tasks including scientific coding, literature review, domain question answering, hypothesis generation, experimental planning, outcome prediction, peer review, and paper production.PaperOrchestra transforms raw materials into submission-ready papers with generated visuals and literature synthesis.Source: §4 · p. 32

End-to-end computational research frameworks

• Existing autonomous research systems span literature review, experimentation, manuscript writing, traceable data-to-paper workflows, and self-improving agent architectures.Agent Laboratory and The AI Scientist execute computational research workflows, while data-to-paper links results to code and data and ADAS discovers agent architectures.Source: §4 · p. 32
• Computational discovery systems increasingly generate and verify novel methods, algorithms, proofs, and biomedical or materials findings through Bayesian optimization, code evolution, and coordinated multi-agent experimentation.DeepScientist surpassed human-designed state-of-the-art baselines across multiple frontier AI tasks, while related systems extend discovery into biology and physical sciences.Source: §4 · pp. 32–33

Systemic limitations and the execution gap

• Autonomous research systems face an ideation-execution gap involving methodological rigidity, reward hacking, technical infeasibility, hallucinations, fabricated code, plagiarism, data leakage, and selection bias.Purely software-based AI Scientists have reported verified plagiarism rates up to 24%, while simulated environments such as DISCOVERYWORLD and MADE benchmark end-to-end discovery under controlled conditions.Source: §4 · p. 33

Bridging the gap: domain-specific discovery and lab-in-the-loop

• Lab-in-the-loop systems bridge computational prediction and physical validation through robotic experimentation, Bayesian optimization, microscopy, and active learning across chemistry, materials, proteins, and drug delivery.Coscientist, A-Lab, LLM-RDF, and ChemCrow connect LLM reasoning to laboratory interfaces, while MULTI-evolve and LUMI-lab incorporate continuous experimental feedback into model-driven discovery.Source: §4 · pp. 33–34

Towards pragmatic human-AI collaboration

• Because fully unsupervised physical execution remains limited by hardware complexity and anomalies, current systems emphasize human-guided collaboration between reasoning models, laboratory infrastructure, and researchers.The Co-Scientist paradigm supplies high-level constraints and domain context, while Virtual Lab and LabOS demonstrate experimentally validated or real-time human-AI assistance.Source: §4 · p. 34

5. Discussion

• Co-Scientist unifies experiment design, code generation, and hardware control with domain-adaptive autonomy and human oversight, advancing closed-loop scientific research.Applications covered materials synthesis, biological phenotype prediction, autonomous architecture discovery, and reliability-focused manuscript generation.Source: §5 · p. 35

5.1. Limitations and failure modes

• Generalization remains unestablished across laboratories, genetic circuits and species, multi-turn clinical settings, and evaluation regimes vulnerable to proxy optimization.Materials recipes were validated only on a custom CVD system; biology showed circularity divergence; and Agent_H was evaluated on single-turn rubrics.Source: §5.1 · pp. 35–36
• Observed failures required intervention, including biologically implausible image artifacts, reward hacking through excessive response length, and Agent_H architectures requiring 40–80 LLM calls per query.Adding a length penalty substantially reduced HealthBench scores, indicating that earlier gains partly reflected verbosity rather than clinical quality.Source: §5.1 · p. 36
• Reliability modules reduced hallucination and plagiarism but remain unproven for noisy physical experiments, cannot eliminate harmful plans, and do not address all methodological biases.Safety filters refuse 98.7% of harmful prompts and redirect over 96% of hazardous directions, yet residual risk remains.Source: §5.1 · p. 36

5.2. Ethical considerations

• Actionable experimental protocols create compositional dual-use risks, while model-driven ideation may narrow scientific inquiry through biased and overly homogeneous hypotheses.The discussion calls for trajectory-level compositional safety analysis and recognizes that novelty objectives and diversified sampling may not ensure conceptual novelty.Source: §5.2 · p. 37
• Autonomous research requires explicit human accountability, institutional governance, and future systems for multimodal verification, recursive improvement, and multi-agent scientific collaboration.Existing review and biosafety frameworks do not adequately address autonomous systems, motivating auditing standards, reporting protocols, and cross-institutional oversight.Source: §5.2 · p. 37

• Co-Scientist advances AI-assisted research from computational ideation toward execution-grounded discovery across materials science, biology, and computer science.The system interfaces with physical laboratory hardware and code execution environments.Source: §6 · p. 38
• Deterministic verification and expert human oversight provide a practical framework for augmenting scientists across adaptive autonomy levels without compromising safety or scientific integrity.Source: §6 · p. 38
• Grounding AI in physical and empirical reality remains a critical bottleneck, making human-in-the-loop collaboration necessary for real-world validation.The appropriate AI role depends on each experimental surface’s physical, safety, and verification demands.Source: §6 · p. 38
• Integrating self-improving discovery agents with automated laboratories and collaborative multi-agent networks offers a scalable path toward closed-loop scientific discovery.The envisioned future is one where validated-discovery pace is bounded by experimental throughput rather than scientific ideation.Source: §6 · p. 38

Data Availability

• HealthBench, HealthBench Professional, and ground-truth bacterial swarm images are publicly available through Hugging Face and Zenodo, respectively.The datasets are identified as openai/healthbench, openai/healthbench-professional, and Zenodo record 19612563.Source: Data Availability · p. 39

Code Availability

• The full Co-Scientist source code is unavailable because of proprietary infrastructure, computational demands, and safety concerns, while experimental access is offered through Google Labs.The released software and tools are research-only prototypes and require independent verification before practical use.Source: Code Availability · p. 39

Competing Interests

• The study was funded by Alphabet Inc. and/or its subsidiaries, and some Alphabet employee authors may own company stock through standard compensation.Source: Competing Interests · p. 39

A. Additional Details on Co-Scientist

• The appendix provides additional details on the extended Co-Scientist system beyond the methodology described in Section 2.Source: A. Additional Details on Co-Scientist · p. 56

A.1.1. Ideation details

• The ideation module converts research directives into testable hypotheses using five specialized agents: Generation, Ethics Review, Reflection, Ranking, and Evolution.Generation creates a stochastic candidate pool augmented by literature retrieval, while each hypothesis stores text, identifiers, lineage, critiques, and a Bayesian skill rating.Source: §A.1.1 · p. 56
• Each evolutionary generation evaluates hypotheses for dual-use risk, novelty, feasibility, and testability, while pairwise LLM tournaments update Gaussian skill ratings through TrueSkill.Source: §A.1.1 · p. 56
• After evaluation and selection, UCB-based tournament sampling balances exploitation and exploration, while crossover and reflection-guided mutation generate offspring until the top candidate advances to experimentation.The UCB score combines estimated quality μ_h with uncertainty σ_h, and reproduction uses crossover probability p_c versus reflection-guided mutation probability 1 − p_c.Source: §A.1.1 · p. 56

A.2.1. Experimentation details

• Resource-aware context aligns experimental designs and parallelization with available CPUs, GPUs, VRAM, memory, and installed environments, preventing out-of-memory and dependency failures.The system incorporates host-environment specifications directly into the agent’s context.Source: §A.2.1 · p. 56
• A staged protocol validates code, data loading, and dependencies on minimal data with 600-second timeouts before replacing scaffolding artifacts and executing full datasets.The transition phase verifies that mock behaviors and subsampling variables are removed.Source: §A.2.1 · p. 56

A.2.2. Addressing infeasible plans

• Dynamic plan reflection synchronizes research plans with executed code by revising infeasible assumptions from execution logs and runtime traces, preventing manuscripts from describing intended rather than executed methods.This addresses failures involving runtime constraints, incompatible APIs, and negative intermediate results (Schmidgall and Moor, 2025; Schmidgall et al., 2025; Si et al., 2024).Source: §A.2.2 · p. 58

A.3.1. Advancements in paper writing

• Co-Scientist dynamically composes manuscript structures from research outcomes, supporting specialized sections, diverse scholarly formats, LaTeX management, cross-references, and citations.It can accommodate interleaved methods-results structures and lab-notebook styles.Source: §A.3.1 · p. 58
• Vision-enabled iterative self-reflection generates figures from experimental code, classifies quality, critiques deficiencies, and retains the highest-scoring outputs for manuscript writing.The loop combines executable Python generation, visual criticism, and VLM scoring until completion or an iteration limit.Source: §A.3.1 · p. 59

A.3.2. Encouraging transparency during experimentation.

• Verbose logging of intermediate variables, statistical summaries, and error traces supports hallucination clipping and prompts targeted logging when outputs lack sufficient information.Sparse or empty execution logs can otherwise encourage fabricated results (Schmidgall et al., 2025).Source: §A.3.2 · pp. 59–60

A.3.3. Preventing harmful code execution

• A mandatory two-stage safety gateway classifies candidate code for harmful intent and sanitizes flagged logic into C′ while preserving the original research objectives.This addresses the inability of standard sandboxing to interpret semantic risks across multi-step plans (Changjiang et al., 2025; Inan et al., 2023; Miculicich et al., 2025; Rebedea et al., 2023; Xu et al., 2023).Source: §A.3.3 · p. 60, Figure A4 · p. 60

B.1. Chemical vapor deposition methods for targeted MXene growth

• CVD used mixed C2Cl6 and Ti powder in an alumina boat with cleaned Ti-foil substrates positioned across a 950–300°C temperature gradient.Source: §B.1 · p. 60
• The quartz tube was purged with 200 sccm high-purity argon for 15 minutes before heating to the 950°C growth temperature.Source: §B.1 · pp. 60–61
• Growth was maintained for 1.5 hours using 200 sccm Ar plus 50 sccm forming gas during ramping and 50 sccm each of Ar and forming gas during growth.Source: §B.1 · p. 61
• Cleaning the tube, o-rings, outlet tubing, quartz tube, and boat limited dust, back-flow contamination, and residual carryover between experiments.Source: §B.1 · p. 61

B.2. Minimally intensive layer delamination (MILD) of the obtained 2D crystals

• After MILD treatment, SEM-EDS detected Ti, C, O, and Cl, with possible F surface terminations and no detectable N in the layered structures.Source: Figure A5 · p. 61
• MILD etching used 20 mL 9 M HCl with 1 g LiF, soaking Ti-foil-supported crystals under magnetic stirring at 35°C for 24 hours before drop casting.Source: §B.2 · p. 62

Structural and chemical characterization

• Characterization combined XRD for crystalline structure, Raman spectroscopy, and XPS analysis of the synthesized 2D crystals.XRD used a Cu source; Raman measurements used a 633 nm laser; high-resolution Ti 2p XPS scans used 20 eV pass energy with 0.1 eV steps.Source: §B.3 · p. 62

2D material transfer

• For transfer, scratching Ti foil in isopropyl alcohol dispersed layered flakes that were deposited onto target substrates and dried for 10 minutes.The Ti surface was first scratched to remove floating black byproducts, and the dispersion was collected with a dropper.Source: §B.3 · p. 62

Electron microscopy

• SEM and EDS examined transferred flakes on SiO2(90 nm)/Si substrates to assess morphology and elemental composition.SEM operated at 2.0 kV and 25 pA, while EDS operated at 20 kV and 0.8 nA.Source: §B.3 · p. 62
• For STEM, flakes were deposited on lacey-carbon copper grids, cleaned to reduce hydrocarbon contamination, and imaged by HAADF-STEM with STEM-EDS at 300 kV.EDS images were filtered by net count intensity, and background-subtracted peak areas were average-filtered to improve spatial signal-to-noise ratios.Source: §B.3 · p. 62

Structural and chemical characterization

• STEM imaging with EDS mapped Ti, C, Cl, N, and O in the 2D flakes, while Raman and Ti 2p XPS characterized the as-grown material.Figure A6 summarizes STEM/EDS mapping, Raman spectra, and Ti 2p XPS spectra for yield analysis.Source: §B.3 · p. 62, Figure A6 · p. 63

B.4. Chemical vapor deposition methods for TMDs

• TMDs were grown in a single-zone tube furnace on cleaned 3.7 cm × 1.7 cm SiO2(300 nm)/Si substrates.Substrates were cleaned with DI water, acetone, and IPA for 5 minutes each, then dried under nitrogen gas.Source: §B.4 · p. 63
• MoS2 CVD used MoO3 and NaCl in a source boat with sulfur powder in a separate upstream boat.The source boat measured 50 mm × 12 mm × 10 mm, while the upstream boat measured 75 mm × 15 mm × 10 mm.Source: §B.4 · p. 63
• MoSe2 CVD used MoO3 and NaCl in a source boat with selenium powder in a separate upstream boat.Both boats had the same dimensions and capacities as those used for MoS2 growth.Source: §B.4 · p. 63
• WS2 CVD used WO3 and NaCl in a source boat with sulfur powder in a separate upstream boat.The source and upstream boats measured 50 mm × 12 mm × 10 mm and 75 mm × 15 mm × 10 mm, respectively.Source: §B.4 · p. 63
• The model generated all growth parameters, including precursor amounts, gas flow, temperature programming, and boat–substrate arrangement.Between runs, the quartz tube and boats were washed with DI water and heated at 1000 oC for at least 50 min to remove residual material.Source: §B.4 · p. 63

B.5. Characterization methods for TMDs

• TMD characterization relied mainly on optical techniques, beginning with autonomous optical microscopy to assess morphology and sizes.The microscope used a Python-based interface with a Zeiss AxioScope 7 and Zeiss Axiocam 705 camera (Yang et al., 2025b).Source: §B.5 · p. 64
• Raman spectra were acquired using a Horiba Jobin Yvon LabRam ARAMIS spectrometer with a 442 nm laser.Source: §B.5 · p. 64

• 99.8% of 1,282 training queries had maximum similarity below 0.75 with either benchmark, and zero exceeded 0.8, confirming that evaluation questions were held out.Training queries were short, patient-facing, non-diagnostic questions, unlike the complex expert-level diagnostic scenarios in HealthBench; mean maximum similarity was 0.38 for Professional and 0.41 for Hard.Source: §C.1 · p. 64
• Rubric overlap was higher for HealthBench Hard than Professional, with mean maximum similarity of 0.72 versus 0.499 and 52.3% versus 5.7% matching at ≥0.70 cosine similarity.For Hard, 7.5% of training rubrics matched at ≥0.9, while Professional showed only 5.7% matching at ≥0.70.Source: §C.1 · p. 64
• Zero exact matches and mean response similarity of 0.748 on Hard and 0.713 on Professional were comparable to frontier models without training-corpus access, indicating no response-level leakage.Table A1 reports these cosine-similarity results across evaluated models and both benchmarks using the Universal Sentence Encoder.Source: §C.1 · p. 64, Table A1 · p. 65

C.2. Autorater agreement analysis

• ρ=0.869 (p<0.0001) measured agreement between judges on per-query performance gaps between Agent_H and Gemini 3.1 Pro across 106 clinical queries.The queries comprised 51 HealthBench Hard and 55 HealthBench Professional cases.Source: §3.5 · p. 65

3.5 Flash

• ρ=0.987 (p=7.38 × 10−11) was the pooled rank correlation across 14 model-benchmark pairs, while both judges placed Agent_H and GPT-5.6 Sol among the top systems under length adjustment.Model-level correlations were ρ=0.893 on HealthBench Hard, ρ=1.000 on Professional raw scores, and ρ=0.929 under length adjustment.Source: §3.5 · p. 65
• κ=0.034–0.243 indicated slight-to-fair agreement between both autoraters and human clinicians across nine evaluation dimensions, with highest agreement on clinical reasoning and inaccurate or irrelevant information.Agreement was lowest for demographic bias evidence, reading comprehension, and information omission; GPT-5.4 Low Reasoning also scored 5–9 points lower on Hard and 2–3 points lower on Professional while preserving relative rankings.Source: Table A2 · p. 66, §3.5 · p. 66

D.1. Expert recruitment / D.3. Human evaluation & labeling guidelines

• 30 domain experts, including 29 with a Ph.D. or post-doctoral position, reviewed generated ideas, plans, manuscripts, logs, and code in a double-blind evaluation.The cohort had mean and median experience of 11 years, spanning 0–23 years, and used external literature-search tools under a standardized rubric.Source: §D.1 · p. 66, §D.3 · p. 68

D.2. Computational environment and configuration

• Experiments ran on two NVIDIA A100 40GB GPUs with isolated execution environments, deterministic logs, and configurable 600-second scaffolding and 18,000-second solver timeouts.This setup excludes research requiring large-scale distributed training, multi-node parallelism, or hardware beyond standard GPU instances.Source: §D.2 · p. 67, Table A4 · p. 67

D.3.1. Ideation evaluation (Quality & Safety) / D.3.2. Experiment plan evaluation (Safety)

• The evaluation rubric rated ideation quality on a 1–5 scale and assessed whether ideas or experiment plans could cause substantial harm, including potential misuse.Risk profiles were categorized as clearly malicious, dual-use/high-risk, or safe/standard research.Source: §D.3.1 · p. 68, §D.3.2 · p. 69

D.3.3. Hallucination study A: Results validation

• Reviewers validated every reported Results claim against experimental logs and scored hallucination severity from 1 to 10, covering cosmetic errors through systemic fabrication.The rubric distinguishes low severity (1–3), moderate severity (4–6), and high severity (7–9) hallucinations.Source: §D.3.3 · pp. 69–70

Total Fabrication (Score 10)

• Score 10 denotes total fabrication: all reported results are fabricated and invalid, including physically impossible experiments, fake units, or contradictions of basic scientific laws.This is the rubric’s maximum-severity category beyond systemic fabrication at score 9.Source: §D.3.3 · p. 70, Total Fabrication (Score 10) · p. 70

D.3.4. Hallucination study B: Methodology & Code validation

• Methodology validation cross-checks every reported algorithm, architectural component, hyperparameter, and data-processing pipeline against the provided Python implementation.A mismatch is classified as a hallucination, with follow-up description and severity scoring when applicable.Source: §D.3.4 · p. 70
• Hallucination severity ranges from 1–3 for minor discrepancies, 4–6 for reproducibility-compromising omissions, and 7–9 when the claimed novelty is unsupported by the code.Examples include mismatched hyperparameters, missing regularization, altered architecture, absent novelty modules, and systemic divergence between paper and implementation.Source: §D.3.4 · pp. 70–71

Total Fabrication (Score 10)

• Score 10 denotes total fabrication: the methodology is computationally impossible or depends on nonexistent libraries, while the code is empty, unrelated, or completely misaligned.This category requires zero meaningful alignment between the Methods section and implementation.Source: Total Fabrication (Score 10) · p. 71

D.3.5. Plagiarism study

• The plagiarism study searches the title, abstract, introduction, and especially Methods against existing literature, then records whether plagiarism is present.OpenScholar and related scholarly search engines are suggested for identifying prior work.Source: §D.3.5 · p. 71

(SKIP IF SELECTED NO to TASK 1) Task 2: Plagiarism Scoring

• Plagiarism scoring ranges from 5 for one-to-one copying to 1 for completely novel findings, with intermediate levels covering mixed, similar, or slightly resembling methods.The evaluation also checks whether heavily borrowed source work is cited and requests the most similar article’s PDF.Source: (SKIP IF SELECTED NO to TASK 1) Task… · pp. 71–72

D.3.6. Code quality evaluation

• Code quality evaluation inspects the main execution script and all helper files against standards expected of a graduate-level research assistant.The assessment covers implementation quality beyond paper–code consistency.Source: §D.3.6 · p. 72

Metric A: Readability & Documentation

• Readability and documentation are scored from 1 for uncommented, opaque code to 3 for documented functions, clear variables, and comments on complex logic.The rubric specifically considers comments, variable naming, docstrings, and explanatory coverage.Source: Metric A: Readability & Documentation · p. 72

Metric B: Modularity & Architecture

• Modularity and architecture range from 1 for an inseparable monolithic script to 3 for decoupled data loading, modeling, and training components.The highest score reflects reusable functions and clear separation of concerns.Source: Metric B: Modularity & Architecture · p. 72

D.4.1. Low severity result hallucination

• Failure rates differed significantly across groups (χ2 = 53.0, p < 3.1×10^-12).Source: §D.4.1 · p. 72
• 22% of Co-Scientist articles using reliability modules contained result hallucinations, versus 54% for ablated Co-Scientist and 94% for Agent Laboratory.The reliable configuration had lower hallucination rates than both alternatives, with adjusted p-values < 0.006 and < 1.6 × 10^-13, respectively.Source: §D.4.1 · p. 72
• Ablated Co-Scientist also reduced hallucinations relative to Agent Laboratory (padj < 2.0 × 10^-5).Source: §D.4.1 · p. 72

D.4.2. Methodological hallucination rates and severity

• Methodological hallucinations occurred in 50% of reliability-enabled Co-Scientist manuscripts, versus 66% for ablated Co-Scientist and 100% for Agent Laboratory.The reliability-enabled condition also had the lowest mean severity score: 2.18 ± 2.41, compared with 4.62 ± 2.85 and 8.34 ± 2.11, respectively.Source: §D.4.2 · p. 73
• Reliability-enabled Co-Scientist significantly lowered methodological hallucination severity and increased methodological integrity relative to both comparison conditions.Pairwise Mann-Whitney U tests with Bonferroni correction found p_adj < 0.015 versus ablated Co-Scientist and p_adj < 10^-16 versus Agent Laboratory.Source: §D.4.2 · p. 73

D.4.3. Low severity plagiarism

• Derivative content appeared in 30% of reliability-enabled Co-Scientist articles, compared with 56% for ablated Co-Scientist and 80% for Agent Laboratory.The reliable configuration had lower derivative-content rates than both alternatives under Fisher’s Exact tests with Bonferroni correction.Source: §D.4.3 · p. 73
• The ablated Co-Scientist system did not significantly reduce derivative content relative to Agent Laboratory (p_adj = 0.053).By contrast, reliability-enabled Co-Scientist differed significantly from ablated Co-Scientist (p_adj = 0.045) and Agent Laboratory (p_adj < 3.0 × 10^-6).Source: §D.4.3 · p. 73

D.5. Inter-rater agreement

• Safety evaluation inter-rater reliability was assessed with Cohen’s Kappa for binary judgments by pairs of independent raters.The agreement results are summarized in Table A5.Source: §D.5 · p. 73, Table A5 · p. 73
• Co-Scientist Safety achieved moderate agreement with κ = 0.38–0.43, reflecting difficulty adjudicating dual-use research directions.Reasonable experts may disagree on these cases, making safety classification inherently challenging.Source: §D.5 · p. 73
• The Ablation condition achieved higher agreement with κ = 0.63–0.80 because its unconstrained outputs were more overtly harmful and easier to classify.This higher agreement was therefore expected rather than evidence of better safety evaluation performance.Source: §D.5 · p. 73

D.6. Qualitative analysis of autonomous research failure modes

• Unconstrained baseline agents fabricated research artifacts, manipulated evaluations with deceptive code and biased settings, and repackaged published frameworks without attribution.Observed failures included invented p-values and t-tests, hardcoded benchmark outputs, duplicated toy datasets, deterministic mock evaluators, and uncited architectures.Source: §D.6 · p. 74
• Co-Scientist eliminated extreme fabrication (severity ≥8) and reduced invalidating result hallucinations to 4%, but residual failures included selective reporting, formula–implementation divergence, and subconscious plagiarism at 16% (severity ≥3).Proposed safeguards include run-completeness audits, symbolic code-to-text alignment, static analysis for mock stubs, and live-retrieval literature verification.Source: §D.6 · p. 75

E. Co-Scientist Task Prompts

• The task prompts constrain MXene synthesis to a fixed single-zone furnace and available chemicals while requiring exact, reproducible, non-hazardous experimental parameters.The objectives target Ti3C2Cl2 growth, one carbon source, necessary chemicals only, and no TiCl4 or HF.Source: E. Co-Scientist Task Prompts · pp. 75–76

2. Output

• The proposed MXene recipe uses 950 °C growth under pure argon, in-situ TiH2-derived hydrogen, and upstream C2Cl6 pseudo-Knudsen delivery to avoid nitrogen poisoning and hazardous liquid precursors.The design places 500.0 mg C2Cl6 at -250.0 mm and uses a 450 °C forming-gas stage before argon purging.Source: §2 · pp. 76–77

3. Our Modification

• The MXene modification mixed Ti powder with C2Cl6, added 50 sccm forming gas during growth, reduced Ti powder to 100 mg, simplified ramping, and extended growth to 1.5 h.These changes were intended to alter precursor chemistry, simplify processing, and increase synthesis yield.Source: §3 · p. 77

1. MoS2 Monolayer Synthesis

• The MoS2 recipe uses 5 mg MoO3, 1 mg NaCl, and 200 mg sulfur, with a 780 °C dwell and programmed argon flow and temperature ramps.MoO3 and NaCl are placed at the furnace center, sulfur at 210 mm upstream, and the substrate faces down over the metal source.Source: §1 · p. 78

2. WS2 Monolayer Synthesis

• The WS2 recipe uses 30 mg WO3, 5 mg NaCl, and 400 mg sulfur, with an 820 °C dwell using 80 sccm argon and 20 sccm hydrogen.The sulfur boat is positioned 215 mm upstream, while the metal source and facedown substrate are centered in the hot zone.Source: §2 · p. 79

3. MoSe2 Monolayer Synthesis

• The MoSe2 recipe uses 5 mg MoO3, 1 mg NaCl, and 250 mg Se, with a 15-minute 800 °C dwell under 60 sccm argon and 15 sccm hydrogen.Selenium is placed 185 mm upstream from the furnace center.Source: §3 · pp. 79–80
• Colony predictions are evaluated by visual fidelity and quantitative concordance across mean radius, polar eccentricity, circumferential intensity CV, and circularity.Generated and ground-truth colonies use identical segmentation and feature-extraction pipelines, with linear mixed-effects models testing IPTG-dependent trajectories.Source: §3 · p. 81
• High-fidelity colony generation combines rich biological context, adjacent-concentration leave-one-out interpolation, Best-of-N rejection sampling, and multi-scale concentration-aware prompting.The pipeline also preserves control-strain invariance and matches the resolution, color depth, and visual style of experimental scans.Source: §3 · pp. 80–81

Medical response generation

• The medical agent must use configurable LLM calls and clinical-guideline retrieval while handling single- and multi-turn conversations, multiple languages, and concise responses.Development uses 1,282 synthetic health queries, including approximately 55% underspecified cases, with held-out benchmarks unavailable during development.Source: §3 · p. 82
• High-scoring medical-response systems deliberately allocate inference-time compute across query types under physician-authored rubrics penalizing omissions, fabrication, incorrect premises, and unsafe advice.The optimization loop iterates between generating responses, rubric-based evaluation, error analysis, and architecture revision.Source: §3 · pp. 82–83