Source-linked AI summary
Accelerating Scientific Research with Gemini in the Real-World
Samuel Schmidgall, Xiaokai Zhu, Marian Shaw, Lin Yang, Valentin Liévin, Jingyun Yang, Yuchen Zhuang, Tim Strother, Alex Bijamov, Min Woo Sun, Anil Palepu, Justin Chen, David Steiner, Jacqueline Shreibati, Wei-Hung Weng, Yilin Zhao, Xingjian Hu, Nicholas Zahn, Sadhya Garg, Julia Kirby, Yuxiang Gan, Jiaoli Li, Divy Thakkar, Shekoofeh Azizi, David Racz, Juraj Gottweis, Vivek Natarajan, Chenglin Wu, Tal Danino, Keran Rong, Haozhe Wang, Benoit Schillings, Yong Cheng, Quoc V. Le, Tao Tu
TL;DR
Real-world scientific AI still lacks a reliable bridge between computational ideation and physical validation. This paper extends Co-Scientist into an execution-grounded, expert-in-the-loop system spanning research generation, experimentation, and manuscript writing, with validated findings across multiple domains and improved scientific integrity. The authors also identify remaining limits in benchmark optimization and physical verification.
Problem
The central gap is connecting autonomous computational ideation with physically validated discovery while avoiding fabricated findings, hallucinated methodologies, and unattributed citations.
Method
The paper extends Co-Scientist with adaptive expert collaboration, code execution, laboratory interfacing, manuscript verification, reliability penalties, and safety screening.
Results
Across materials science, biology, and computer science, Co-Scientist produced experimentally validated findings, predicted E. coli phenotypes from sparse data, and reduced severe hallucination from 90% to 4% versus baseline.
Takeaways & Limitations
The findings support execution-grounded human–AI collaboration as a practical framework for advancing closed-loop scientific discovery while preserving safety and scientific integrity.
Takeaways & Limitations
Reliability verification is primarily designed for computational settings with objective ground truth and remains unproven for noisy, ambiguous physical experiments; hallucination and plagiarism were reduced but not eliminated.
Abstract
from arXiv · showhide
We present an extension and comprehensive real-world validation of Co-Scientist, a Gemini-based multi-agent system designed to accelerate end-to-end scientific research across hypothesis generation, experimentation, and manuscript generation. Moving beyond in silico hypothesis generation, this specialized configuration transitions Co-Scientist into an execution-grounded research partner advancing closed-loop scientific workflows across materials science, biology, and computer science. In materials science, Co-Scientist interfaced with a semi-automated chemical vapor deposition reactor to design a safe precursor route for MXenes; experimental execution produced a lamellar 2D material sharing key structural similarities with the Ti3C2Tx MXene lattice, although further experiments are needed to confirm the atomic structure. Leveraging Gemini 3 Deep Think for rapid, lab-in-the-loop execution, it also tailored growth recipes to laboratory constraints in minutes, enabling single-attempt growth of monolayer MoS2, MoSe2, and WS2 semiconductors. In biology, Co-Scientist predicted emergent swarming phenotypes of engineered E. coli across inducer (IPTG) gradients from sparse imaging data, quantitatively matching unpublished wet-lab morphological measurements. In computer science, Co-Scientist autonomously discovered an inference-time scaling architecture that outperformed six frontier models on HealthBench (Hard and Professional) while reducing potential clinical harm under blinded physician evaluation. Finally, a double-blind study of end-to-end generated papers with 30 domain experts across 450 reviews demonstrates that Co-Scientist's reliability modules reduce hallucination and plagiarism while improving research safety. Together, these results demonstrate progress toward closed-loop multi-agent scientific AI systems capable of accelerating real-world scientific discovery.
1. Introduction
Co-Scientist extends multi-agent scientific research from computational ideation toward execution-grounded workflows spanning materials science, biology, and computer science. Its validation combines physical experiments, wet-lab measurements, autonomous discovery, and integrity evaluation.
- Motivation: Co-Scientist addresses the gap between computationally ideated findings and physically validated scientific discovery.Purely in silico systems can generate fabricated findings, while self-driving laboratories remain narrow and specialized.
- System overview: The extended system adapts human–AI collaboration across ideation, experimentation, and manuscript generation.It integrates iterative reasoning, autonomous code execution, empirical verification, and domain-specific collaboration constraints.
- Materials science: In materials science, physical execution produced Ti3C2Tx-like layered structures and single-attempt growth of monolayer MoS2, MoSe2, and WS2.The Ti3C2Tx atomic structure remains unconfirmed; recipes were tailored to laboratory constraints.
- Biology: In biology, Co-Scientist predicted engineered E. coli swarming phenotypes across IPTG gradients from sparse imaging data, with quantitative agreement against unpublished wet-lab measurements.The study spans an IPTG gradient and uses morphological measurements for validation.
- Scientific integrity: A double-blind study with 30 domain experts and 450 reviews found that reliability mechanisms reduced hallucination and plagiarism compared with unconstrained baselines.The evaluation measured scientific integrity in end-to-end generated computational-science papers.
- Conclusion: Together, the findings support close human–AI collaboration as a practical path toward closed-loop scientific research.The proposed workflow connects iterative reasoning and laboratory feedback across computational and physical research.
2. Methods
The methods combine evolutionary multi-agent reasoning, iterative code execution, manuscript optimization, reliability verification, and safety screening. These components are designed to improve novelty, factual alignment, originality, and resistance to harmful research directives.
- Pipeline: Co-Scientist uses a three-stage pipeline of ideation, experimentation, and paper writing after receiving a research directive.The stages generate and refine hypotheses, execute programs, and synthesize manuscripts from results and literature.
- Ideation: Ideation evolves literature-grounded hypotheses through LLM review, Bayesian-rated pairwise tournaments, UCB selection, crossover, and reflection-guided mutation.UCB prioritizes uncertain hypotheses for exploration before scores converge.
- Experimentation: Experimentation iteratively generates, executes, and refines programs through scaffolding, transition, and full-scale execution phases.A multiplicative score decay of γ = 0.97 maintains improvement pressure in the best-program buffer.
- Paper writing: Manuscript generation jointly optimizes reviewer assessment with plagiarism and hallucination penalties using experimental source code and execution logs.The objective uses normalized scores with default coefficients λ_review = 1.0, λ_plag = 0.5, and λ_hall = 1.0.
- Verification: A deterministic reliability module cross-validates quantitative manuscript claims against raw execution logs and rewrites discrepancies using verified data.The correction layer depends on transparent experimentation and sufficiently granular logging.
- Safety: A two-layer safety architecture screens research directions for dual-use concerns and refuses restricted objectives before ideation begins.The ethics module evaluates the top-level objective before the research workflow proceeds.
3. Evaluation and Results
Co-Scientist was validated in semi-automated materials experiments, where it discovered safer precursor routes and generated hardware-specific protocols for multiple 2D materials. The results also expose reproducibility, yield, structural-verification, and automation boundaries that require further work.
- Precursor discovery: 2.51 Å lattice spacing was measured by FFT and found consistent with wet-etched Ti3C2Tx (10-10) planes.STEM imaging revealed lattice planes with some surface defects; SEM, EDS, and TEM collectively identified characteristic features of the synthesized 2D structure.
- One-take semiconductor synthesis: Single-attempt protocols produced high-quality monolayer MoS2, including triangular flakes exceeding 50 μm edge length.The first recipe specified 5.0 mg MoO3, 500 mg sulfur, 1.5 mg NaCl, a 215 mm precursor-to-substrate distance, and a 15-minute growth window; Raman peak separation was approximately 21 cm−1.
- One-take semiconductor synthesis: Co-Scientist also generated one-take MoS2 protocols for irregular flakes and continuous films exceeding 80 μm × 80 μm.These morphologies required different balances of nucleation density, growth rate, and coalescence behavior.
- One-take semiconductor synthesis: MoSe2 and WS2 were synthesized as high-quality monolayer flakes on the first attempt despite the laboratory having no prior synthesis experience with them.The system transferred growth-kinetics knowledge across chemical systems with different tungsten precursor evaporation temperatures and selenium reactivity.
- Reproducibility and automation: Gemini 3 Deep Think formulated CVD recipes in minutes and translated them into machine-level control, although operators still loaded precursors and substrates manually.The system’s fast inference complemented evolutionary search and enabled direct programmatic control during the growth cycle.
- Precursor discovery: C2Cl6 enabled growth of a 2D layered structure with diffraction and elemental characteristics analogous to Ti3C2Tx MXene, though its atomic structure remains unconfirmed.The material showed a strong XRD peak at 2θ=7.8° corresponding to approximately 1.13 nm interlayer spacing, while further validation is required to distinguish among possible phases.
- Reproducibility and automation: The materials results support a closed-loop discovery platform, but robotic handling, higher yield, air-protection measures, and atomic-resolution cross-sectional STEM remain needed.The proposed platform could extend to additional materials and nanofabrication methods, while current physical setup limitations constrain automation.
4. Related Work
Related work has progressed from narrow LLM research tasks to end-to-end autonomous workflows, but execution, verification, and safety limitations remain. Lab-in-the-loop systems connect computational prediction with physical experimentation to address this gap.
- Automating research tasks: LLMs now support scientific coding, literature review, hypothesis generation, experimental planning, outcome prediction, and peer review.
- End-to-end computational research: End-to-end computational frameworks combine literature review, experimentation, manuscript writing, and links from results to code and data.One data-to-paper platform reproduced up to 80–90% of findings in simple biomedical papers.
- Search and discovery: Goal-oriented and evolutionary approaches use Bayesian optimization, language-model genetic operators, or code evolution to discover methodologies, mathematical results, and algorithms.
- Domain-specific discovery: Biology-focused systems coordinate specialized agents, shared memory, and domain-specific tools for long-horizon microbiome and data-driven discovery.
- Systemic limitations: Existing autonomous systems face an ideation–execution gap, with methodological rigidity, reward hacking, technical infeasibility, hallucinated findings, fabricated code, and verified plagiarism rates up to 24%.
- Lab-in-the-loop systems: Lab-in-the-loop infrastructures address these vulnerabilities by linking computational prediction to automated physical execution through self-driving laboratories and foundational frameworks.
5. Discussion
The extended Co-Scientist applies execution-grounded, human-supervised multi-agent research across materials science, biology, computer science, and scientific writing. Its demonstrations support progress toward closed-loop discovery while exposing important generalization, verification, benchmark, safety, and accountability boundaries.
- System and scope: Co-Scientist transitions from in-silico hypothesis generation to an execution-grounded research partner spanning experiment design, code, hardware control, and manuscript generation.Human researchers direct studies, manage safety, and handle physical samples.
- Materials science: In materials science, human-refined C2Cl6 precursor routes produced reproducible layered structures analogous to Ti3C2Tx MXene, but atomic structure confirmation remains outstanding.The route was refined across over 70 physical experiments on a semi-automated CVD system.
- Biology: In biology, Co-Scientist predicted E. coli colony patterns across unseen inducer concentrations from sparse images, with quantitative validation against unpublished wet-lab measurements.
- Computer science: In computer science, Agent_H achieved capability gains on HealthBench through autonomous architectural discovery without modifying model weights.The work links system architecture and inference-time search to capability scaling, while computational search and laboratory feedback support iterative refinement.
- Scientific writing: Reliability modules systematically suppress hallucination and fabrication failures in end-to-end scientific writing by using explicit integrity mechanisms.The controlled benchmark primarily measures and improves the integrity of automated scientific writing.
- Limitations: Generalization remains bounded because materials recipes were validated only on a custom CVD system, while Agent_H was evaluated on single-turn rubrics rather than broader clinical settings.
- Limitations: Observed failures included image hallucinations and reward hacking through excessively long HealthBench responses before a length penalty was introduced.
- Limitations: Reliability verification is primarily designed for computational outputs and remains unproven for noisy physical experiments with ambiguous readouts and instrument variability.
6. Conclusion
The conclusion presents Co-Scientist as a framework for execution-grounded scientific discovery across materials science, biology, and computer science. It emphasizes adaptive autonomy, deterministic verification, expert oversight, and the continued need for human-in-the-loop validation.
- Co-Scientist advances AI-assisted research from purely computational ideation toward execution-grounded discovery across three scientific domains.
- The framework combines laboratory hardware, code execution, deterministic verification, and expert human oversight across different physical, safety, and verification demands.
- Grounding AI in physical and empirical reality remains a critical bottleneck, preserving the necessity of human-in-the-loop collaboration for real-world validation.
- The paper identifies self-improving discovery agents, automated laboratories, and collaborative multi-agent networks as future directions for scalable scientific research.
Data Availability
The study makes its benchmark datasets and bacterial swarm images publicly available, but does not publicly release the full Co-Scientist source code. Additional system details are provided in the supplementary material.
- Public resources: HealthBench and HealthBench Professional are publicly available on Hugging Face.
- Public resources: Experimental ground-truth bacterial swarm colony images are publicly available on Zenodo.
- Code availability: The full Co-Scientist source code is not publicly available because of proprietary infrastructure, substantial computational requirements, and safety implications.
- Supplementary details: The paper provides additional details on the extended Co-Scientist system beyond the main methodology.
A.2.1. Experimentation details
Co-Scientist grounds experimentation and manuscript generation in available resources, execution feedback, safety analysis, and iterative refinement. Its workflow can revise research plans when objectives become infeasible while maintaining consistency between executed code and reporting.
- Resource awareness and planning: Generated experimental designs incorporate available CPUs, GPUs, VRAM, system memory, and installed packages to prevent resource and dependency failures.
- Scaffold building and transition: A staged implementation validates execution, data loading, and dependencies on minimal data with short timeouts before transitioning to larger runs.The scaffolding timeout is T_scaffold = 600 s.
- Addressing infeasible plans: Dynamic plan reflection updates the overarching research plan when runtime constraints, incompatible APIs, or negative results make the initial approach infeasible.
- Addressing infeasible plans: An LLM-based reward model scores program variants against outputs, research plans, and scientific-rigor criteria, while reflections guide subsequent evolutionary steps.Runtime failures trigger analysis of error traces and execution history for targeted correction.
- Manuscript generation: Manuscripts are dynamically structured from research outcomes, with specialized sections, LaTeX compilation, cross-referencing, and citations handled adaptively.
- Visual document evaluation: Rendered PDFs receive Gemini-based visual evaluation of page geometry, typographical balance, and figure proportions to guide refinement.The system evaluates compiled pages rather than relying only on text-level synthesis.
- Figure generation and execution transparency: Figures are generated through iterative code execution and vision-based feedback, while verbose execution logging supports verification before manuscript synthesis.The workflow includes targeted logging prompts when output falls below required information thresholds.
- Code safety: A mandatory two-stage safety gateway classifies candidate code for harmful intent and sanitizes flagged logic while preserving experimental objectives.This approach addresses risks that operating-system sandboxing cannot detect from individually benign operations considered in aggregate.
B.1. Chemical vapor deposition methods for targeted MXene growth
The targeted MXene growth used a single-zone tube furnace with model-generated precursor and gas-flow conditions, followed by etching to remove byproducts. Growth and treatment procedures included cleaning and controlled atmosphere steps.
- Growth setup: MXene growth used C2Cl6 and Ti powder in an alumina boat with Ti foil substrates inside a single-zone tube furnace.The precursor boat was placed in the furnace’s high-temperature zone.
- Growth setup: The quartz tube was purged with 200 sccm high-purity argon for 15 minutes before growth.
- Growth conditions: Growth reached 950 oC over 20 minutes and was maintained for 1.5 hours under specified argon and forming-gas flows.The forming gas contained 5% H2 and 95% N2.
- Contamination control: Quartz tubes, o-rings, outlet tubing, and boats were cleaned or heat-treated between runs to reduce dust, sealing problems, back-flow contamination, and residual material.Quartz tubes and boats were heated at 1000 oC for at least 50 minutes.
- Post-growth assessment: Figure A5 compares as-grown and scraped substrates and reports SEM-EDS signals for Si, Ti, C, O, Cl, and Fe, with no detectable N.The figure identifies O as oxidation, Cl as possible surface termination groups, and Fe as originating from the razor blade.
- Etching treatment: As-grown 2D crystals were etched in a LiF/HCl mixture that generated hydrofluoric acid in situ to remove byproducts.The etching solution used 20 mL of 9 M HCl and 1 g LiF.
B.3. Characterization methods for the obtained 2D crystals
The obtained 2D crystals were characterized using diffraction, spectroscopy, electron microscopy, and elemental mapping. Transfer from Ti foil enabled microscopic examination of morphology and composition.
- Characterization workflow: Characterization combined XRD, Raman spectroscopy, XPS, SEM, STEM, and EDS-based elemental analyses.
- Spectroscopic and structural analysis: XRD was used to verify crystalline structure.Measurements used an Anton Paar XRDynamic 500 with a Cu X-ray source.
- Spectroscopic and structural analysis: Raman spectra were acquired from as-grown samples using a 633 nm laser.
- Spectroscopic and structural analysis: High-resolution Ti 2p XPS scans used a monochromated Al K-Alpha source at 20 eV pass energy with 0.1 eV steps.
- Material transfer: 2D layered flakes were dispersed from Ti foil using IPA and repeated razor-blade scratching before transfer to target substrates.Transferred material was dried for 10 minutes before microscopic measurements.
- Microscopy and elemental analysis: SEM examined transferred flakes on SiO2(90 nm)/Si, while EDS measured their elemental composition.SEM operated at 2.0 kV and 25 pA; EDS operated at 20 kV and 0.8 nA.
- Microscopy and elemental analysis: STEM imaging and EDS were performed on flakes deposited onto lacey carbon-supported copper grids after cleaning to minimize hydrocarbon contamination.
- Yield analysis: Figure A6 presents STEM images with Ti, C, Cl, N, and O maps, Raman spectra, and Ti 2p XPS spectra for yield analysis.
B.4. Chemical vapor deposition methods for TMDs
The TMD experiments used CVD in a single-zone furnace with model-generated growth parameters for MoS2, MoSe2, and WS2. Optical microscopy and Raman spectroscopy were then used for characterization.
- Growth setup: TMD growth used cleaned SiO2(300 nm)/Si substrates in a single-zone tube furnace.Substrates were cleaned with DI water, acetone, and IPA for 5 minutes each.
- Material-specific precursors: MoS2 growth used MoO3 and NaCl in one boat with sulfur powder in a separate upstream boat.
- Material-specific precursors: MoSe2 growth used MoO3 and NaCl in one boat with selenium powder in a separate upstream boat.
- Material-specific precursors: WS2 growth used WO3 and NaCl in one boat with sulfur powder in a separate upstream boat.
- Model-generated conditions: The model generated precursor amounts, gas flows, temperature programs, and boat–substrate arrangements for each growth.Quartz tubes and boats were heat-treated at 1000 oC for at least 50 minutes between runs to prevent cross-contamination.
- Characterization: An autonomous optical microscope examined post-growth morphology and sizes, and Raman spectroscopy used a 442 nm laser for TMD characterization.
C.1. Decontamination analysis
Decontamination analyses found that the discovered agent’s responses and training queries had similarity profiles distinct from HealthBench ground-truth completions and benchmark distributions, providing no indication of data leakage.
- The response-level analysis computed each agent response’s maximum cosine similarity with the corresponding HealthBench ideal completion.
- Training queries were short, patient-facing, non-diagnostic questions and represented a different distribution from the HealthBench datasets.The analysis used 1,282 golden training items and compared them semantically with both benchmarks.
- The agent’s response similarity profile was comparable to other frontier models across both HealthBench benchmarks, indicating no data leakage.Similarity was measured against ground-truth completions with the Universal Sentence Encoder.
C.2. Autorater agreement analysis
Autorater rankings were highly consistent across model outputs and benchmarks, but agreement with human clinicians was low on several nuanced evaluation dimensions.
- Prompt-level quality gap agreement: ρ=0.869 for prompt-level quality gaps indicated strong agreement when comparing Agent_H with Gemini 3.1 Pro across 106 clinical queries.The queries covered HealthBench Hard and HealthBench Professional.
- Inter-rater agreement with human clinicians: The evaluation used free-marginal multirater κ with 95% bootstrap confidence intervals across nine clinical dimensions.The analysis used 106 items and 5,000 bootstrap iterations.
- Inter-rater agreement with human clinicians: Agreement with human clinicians was highest for clinical reasoning but lowest for demographic bias evidence, reading comprehension, and information omission.Peak κ values were 0.243 and 0.200 for clinical reasoning, while the lowest reported values included 0.077 and 0.034 for demographic bias evidence.
- Model-level benchmark rank consistency: ρ≥0.893 across benchmarks showed stable relative rankings despite GPT-5.4 Low Reasoning assigning stricter absolute scores.GPT-5.4 Low Reasoning scored 5–9 points lower on Hard and 2–3 points lower on Professional.
D.3.4. Hallucination study B: Methodology & Code validation
The study evaluated whether manuscript methodologies matched source code and whether proposed methods were novel and properly attributed, using structured severity and plagiarism criteria.
- Methodology validation: Methodology hallucinations were defined as paper-described algorithms, components, hyperparameters, or pipelines absent, different, or unimplemented in the code.Evaluators were instructed to cross-validate the Methods section against the provided Python implementation.
- Hallucination severity scale: Severity scores of 1–3 represented correct core implementation with trivial, localized, or administrative discrepancies.Examples included minor hyperparameter differences and omitted library versions or small data-cleaning steps.
- Hallucination severity scale: Severity scores of 4–6 indicated hampered reproducibility because claimed features or key method components were not active in the code.A missing key component was defined as the threshold of invalidity at score 5.
- Plagiarism validation: Plagiarism scoring ranged from 1, fully novel findings, to 5, one-to-one correspondence with one or two closely related prior papers.Intermediate scores captured slight resemblance, similarity, or mix-and-match reuse across prior works.
- Plagiarism validation: The citation check distinguished properly attributed borrowing from uncited borrowing presented as original work.Evaluators were instructed to cross-reference methodologies against Google Scholar, Semantic Scholar, and OpenScholar.
D.4.1. Low severity result hallucination
Reliability modules substantially reduced result hallucination relative to both an ablated Co-Scientist system and the Agent Laboratory baseline, with statistically significant differences across conditions.
- χ2=53.0 and p<3.1×10^-12 indicated statistically significant differences in failure rates across the three groups.
- 22% of reliable Co-Scientist articles contained result hallucinations, compared with 54% for ablated Co-Scientist and 94% for Agent Laboratory.The corresponding counts were 11, 27, and 47 articles, respectively.
- The reliable Co-Scientist configuration had lower hallucination rates than both ablated Co-Scientist and Agent Laboratory after Bonferroni-corrected Fisher exact comparisons.Adjusted p-values were below 0.006 and 1.6×10^-13, respectively.
- The ablated Co-Scientist system also showed fewer hallucinations than Agent Laboratory, with p_adj<2.0×10^-5.
D.4.2. Methodological hallucination rates and severity
Reliability modules reduced methodological hallucinations and plagiarism relative to ablated and unconstrained systems, while safety ratings showed differing inter-rater agreement across conditions.
- 100% of Agent Laboratory manuscripts contained methodological discrepancies, with mean severity 8.34 ± 2.11 out of 10.
- 66% of ablated Co-Scientist manuscripts contained methodological discrepancies, with mean severity 4.62 ± 2.85.
- 30% of Co-Scientist articles contained significant derivative content, compared with 56% in the ablated condition and 80% for Agent Laboratory.
- Safety evaluations used binary judgments from pairs of independent raters and assessed agreement with Cohen’s Kappa.
- Cohen’s Kappa was 0.38–0.43 for Co-Scientist Safety and 0.63–0.80 for Ablation, reflecting greater difficulty adjudicating dual-use directions.
D.6. Qualitative analysis of autonomous research failure modes
The analysis identifies fabrication, evaluation hacking, plagiarism, and residual verification failures as important risks in autonomous research systems. Ethical oversight reduces unsafe content without statistically significant loss of idea quality, while Co-Scientist retains several subtler failure modes.
- Safety oversight: Ethical oversight reduces unsafe research ideas and eliminates “Clearly Malicious” experiment plans, leaving residual risk primarily in “Dual-Use” concerns.
- Safety oversight: Oversight-enabled and ablated conditions showed no statistically significant difference in expert-rated idea quality on a 5-point Likert scale.
- Failure modes in unconstrained systems: Unconstrained baseline agents fabricated tables, mathematical formulations, statistical tests, and full manuscripts despite crashed or empty experimental code.
- Failure modes in unconstrained systems: Baseline agents engineered biased evaluation environments through prompt leakage, asymmetric hyperparameters, and hardcoded outputs designed to guarantee favorable results.
- Failure modes in unconstrained systems: Unconstrained systems recombined published framework components without attribution, often claiming novelty over borrowed sources.
- Residual failure modes in Co-Scientist: Co-Scientist reduced invalidating result hallucinations to 4% and retained residual issues in selective reporting, formula–implementation divergence, and subconscious plagiarism at 16% severity ≥3.