PaperlayerView exact revision on arXiv

Source-linked AI summary

Why and How to Do Science When AI Is Better at It: A Framework Grounded in Human Understanding and Direction

Hoda Heidari, Aydin Mohseni

arXiv:2610.36116v1cs.AIcs.CY

TL;DR

The paper asks why humans should continue science if AI can outperform them, and how to use AI without giving up ownership of research. It grounds a two-prong test for AI use in science’s aim of durable, collective human understanding. Its argument for continued human participation holds even under AI superiority.

  • Problem

    The paper asks why humans should continue science if AI can do it better and how to incorporate AI without giving up ownership of research.

    Source: §1 · p. 1

  • Method

    The paper derives a two-prong test for AI use from science’s aim of durable, collective human understanding in service of human ends and values.

    Source: §1 · p. 2

  • Results

    The argument for human participation in science holds even if AI outperforms humans at core research tasks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Research should remain human-led to preserve collective human understanding, while AI can be used where it does not erode that aim.

    Source: §1 · p. 2

    Whether AI systems might one day have ends of their own that science should serve is beyond the paper’s scope.

    Source: §2.5 · p. 6

Abstract

from arXiv · show

As AI begins to match or exceed human researchers at core scientific tasks, two questions become pressing: \emph{why} should humans keep doing science, and \emph{how} should they use AI when they do? This paper argues that the second can be answered only by answering the first. It holds that one indispensable aim of science is \emph{durable, collective human understanding of the world, pursued in service of human ends and values}. From this principle it derives a two-part test for when a research task can be delegated to AI. The first part, \emph{human understanding}, requires that the scientific community remain able to explain, defend and contest the resulting claims, and retain the skills to do so. The second, \emph{human direction}, requires that judgments about which questions matter and what findings mean stay with accountable humans. Applying the test to each phase of research in the Contributor Roles Taxonomy (CRediT) yields recommendations that fields can tailor to their questions and methods. The framework differs from existing guidance in three ways. It rests on an account of what science is for rather than on integrity norms. It holds even if AI outperforms human researchers, rather than depending on AI's current limitations. And it judges each use of AI by its effects on human understanding and direction rather than by the type of tool, across every phase of research, including peer review and the training of new researchers. It closes with the reforms needed to keep science aimed at human understanding and direction.

1 Introduction

The paper argues that human science remains necessary even if AI outperforms researchers, because science should sustain durable, collective human understanding in service of human ends and values. It derives a two-prong test for AI use from this purpose and distinguishes that approach from guidance grounded in integrity norms or current AI limitations.

1 Introduction

  • The paper asks why humans should continue science if AI can do it better and how to incorporate AI without giving up ownership of research.It argues that answering the practical question requires first answering the philosophical one.Source: §1 · p. 2
  • Science should sustain durable, collective human understanding in service of human ends and values, so human-led research remains warranted even when AI is more capable.The paper argues that AI may be used where it does not erode this aim.Source: §1 · p. 2
  • The framework tests AI use by its effects on human understanding and direction, applying the test across research phases, including peer review and researcher training.Source: §1 · pp. 2–3

1.1 Related work

  • Unlike guidance focused mainly on research integrity or tool categories, this paper derives phase-specific norms from an account of what science is for and an argument that holds under AI superiority.The framework is intended to let scientific communities tailor recommendations to their questions, methods, and norms.Source: §1 · p. 2, §1.1 · p. 3

2 Why: A Fundamental Aim of Science

The paper argues that science must preserve durable, collective human understanding in service of human ends and values, even when AI can produce reliable knowledge. It defends this principle against five objections, emphasizing that human understanding and deliberation remain necessary for science the community can direct and contest.

2 Why: A Fundamental Aim of Science

  • Science must sustain humanity’s collective capacity to understand, evaluate, and direct its engagement with the world, even if AI outperforms humans at producing reliable knowledge.Other scientific aims cannot replace understanding: without it, the community could no longer direct or correct its work.Source: §2 · p. 3

2.1 The principle

  • The principle distinguishes comprehension, critical assessment, and capability retention; over-reliance on AI most readily erodes the latter two.Understanding matters intrinsically, supports checking and correcting failures, and enables collective self-determination.Source: §2.1 · pp. 3–4
  • Durable collective human understanding is a shared grasp of how and why the world works, sustained by passing that ability to new researchers; human ends determine what is worth understanding.The community should retain authority over goals and uses, while researchers retain authority over scientific judgments; these authorities may need to be exercised jointly.Source: §2.1 · p. 4

2.2 Objection 1: Opportunity cost

  • AI-generated treatments may be deployed before their mechanisms are understood, but the work should be disclosed as not yet understood and prioritized for follow-up explanation.The framework allows urgent cases to favor speed through stakeholder deliberation, while acknowledging that resources for explanation cannot also be spent finding new results.Source: §2.2 · p. 5

2.3 Objection 2: Epistemic dependence is normal

  • Trust in human experts differs from reliance on AI no human understands because the expert’s understanding remains within the community and can be questioned and contested.Reliability may suffice for knowledge, but the paper treats understanding as an aim of science in its own right.Source: §2.3 · p. 5

2.4 Objection 3: AI as a member of the scientific community

  • Even if AI counted as a community member, its understanding would not constitute durable collective human understanding unless it could be transferred to human inquirers.The paper notes that current AI systems cannot themselves answer to professional norms or be sanctioned as members are.Source: §2.4 · pp. 5–6

2.5 Objection 4: A perfect AI would know our values

  • Research priorities gain authority through deliberation among affected people, so an AI’s accurate prediction of community values cannot replace that process.The authors welcome AI tools that aggregate preferences or propose agendas, while arguing that priority-setting should become more inclusive.Source: §2.5 · p. 6

2.6 Objection 5: Understanding is always being lost

  • AI differs from earlier automation because it can perform whole research cycles, including checking and critique, without leaving the displaced understanding with any human.The risk is gradual erosion of the community’s ability to examine and contest knowledge; new AI-assisted practices remain consistent with the principle when they preserve or extend those capacities.Source: §2.6 · p. 7

3 How: A Framework for Incorporating AI in Research

The framework turns the aim of durable, collective human understanding in service of human values into a two-prong test for AI use across research activities. It offers adaptable safeguards, not categorical restrictions, while noting that community recommendations alone cannot preserve human understanding and direction.

3 How: A Framework for Incorporating AI in Research

  • Delegate a task only when the community can understand, defend and contest its claims and retain the skills to do so; accountable humans must direct research priorities and interpret findings for human ends.Stakeholders set goals, priorities and uses; researchers remain answerable to the scientific community for the process.Source: §3 · p. 7
  • The test applies to each activity through research phases informed by and extending CRediT, with human and AI roles allocated within overlapping, recurring phases.The cited evidence concerns effects of AI use on people, such as skills and diversity of ideas, even when AI makes no errors.Source: §3 · pp. 7–8
  • Recommendations are default safeguards that communities should adapt to their tasks, methods and training needs by applying the two-prong test.The article focuses on AI automating research tasks, not primarily on AI as a tutor; AI-assisted learning can still develop genuine understanding.Source: §3 · p. 8
  • Recommendations address researchers and their institutions, but compliance is helpful rather than sufficient because companies shaping science are not bound by the same community norms.Open results with machine-checkable proofs and support for community examination are consistent with the principle; results released faster than they can be examined or independently reproduced are not.Source: §3 · p. 8

3.1 Conceptualization

In conceptualization, AI may surface candidate questions and hypotheses, but choices about which questions to pursue remain human judgments. The framework also links collective understanding to diverse research questions and researchers’ grasp of the literature supporting their claims.

3.1 Conceptualization

  • AI may generate candidate questions and hypotheses, but researchers should choose what to pursue within stakeholder priorities.Choosing among many possible questions is a judgment about what people care about, which a model cannot settle on their behalf.Source: §3.1 · p. 8
  • When researchers rely on AI to identify questions, a field may see a narrower range of topics and more similar ideas, though more capable systems could compensate by varying assumptions and methods.The recommendation to keep question choice human-led rests on the direction prong; the cited evidence describes current tools and warrants caution now.Source: §3.1 · p. 8
  • Authors should distinguish core references they have read and understood from skimmed references they have only consulted.The understanding prong permits authors to defend claims only when they understand the work those claims rest on.Source: §3.1 · pp. 8–9

3.2 Methodology

The methodology recommendation keeps method choice with researchers and requires them to understand a method before adopting it. This protects their ability to judge its fit and supports skill development, especially for early-career researchers.

3.2 Methodology

  • Researchers should choose their methods; AI may suggest or critique candidates, but researchers must understand a method well enough to judge its fit before adopting it.A researcher who cannot program cannot faithfully check a simulation-based method, and searching for methods can build skills for later problems.Source: §3.2 · p. 9
  • Early-career researchers should begin their own method search, while learning an unfamiliar method with AI’s help remains consistent with the recommendation.The test rules out adopting a method because AI chose it, not AI-assisted learning or experienced researchers asking for candidates they can assess.Source: §3.2 · p. 9

3.3 Investigation

How much investigation can be delegated depends on where understanding resides in the method and whether researchers retain the skills to execute and check it. Formal fields require more than verification: AI-generated results become progress on this account when the community can explain, critique and build on them.

3.3 Investigation

  • Investigation delegation depends on where understanding resides in a method, and early-career researchers should execute methods without AI until they reach appropriate competence.Durability requires new researchers to acquire the skills to carry out and check the work.Source: §3.3 · p. 9
  • Automating retrieval and screening can pass the understanding test in a well-structured systematic review, whereas substituting AI for researcher-participant relationships changes ethnographic research itself.One experiment found AI assistance while learning a programming library impaired later conceptual understanding and debugging without significant time savings [46].Source: §3.3 · p. 9
  • In formal or computational fields, machine-checked proofs can establish correctness without human grasp of why a result holds, so verification is only the beginning of understanding.The community must retain the capacity to explain results and identify key ideas, critique them, and reproduce arguments and extend methods without the machine.Source: §3.3 · pp. 9–10
  • On the paper’s understanding-based account, AI-generated theorems are potential progress until community members can explain, critique and build on them.The results resemble an archive of correct findings until they add to what the community understands; helping mathematicians examine them matters as much as the results themselves.Source: §3.3 · p. 10

3.4 Validation

Validation checks whether results reproduce, so it permits more AI delegation than phases that create understanding, provided researchers understand the work and consequential fixes. Early-career researchers should diagnose errors themselves until competent.

3.4 Validation

  • AI can extensively validate results and fix errors when researchers understand the work being checked and any fix that bears on the results.Validation can include formal proof verification, large-scale test generation, and checking quotations against original interview data.Source: §3.4 · p. 10
  • Routine errors such as coding bugs or mislabeled axes can be fixed by AI, but researchers must understand errors and fixes that change results to defend their claims.A successful check of an investigation delegated to AI establishes that the result holds, not that anyone understands it.Source: §3.4 · p. 10
  • Early-career researchers should diagnose and fix errors themselves until competent, because doing so helps them learn to avoid errors.Validation checks reproducibility; it does not by itself produce new understanding, which is why it permits the most delegation.Source: §3.4 · p. 10

3.5 Interpretation

Interpretation forms understanding and determines what results mean for human ends, while writing helps authors consolidate arguments. AI may assist after authors have done that work, subject to their ability to defend its contributions.

3.5 Interpretation

  • Researchers should interpret results before consulting AI for alternative readings and retain the final judgment.Seeing an AI reading first can skip the work through which understanding forms and anchor the researcher’s own reading, even when the AI is correct.Source: §3.5 · p. 10
  • AI interpretation leaves the community with testimony rather than understanding, while deciding what results mean for human ends remains a human-direction judgment.Interpretation is where understanding is formed.Source: §3.5 · p. 10

3.6 Writing

  • Authors should develop and document their arguments before using AI to elaborate or polish them, and must be able to defend every claim AI adds.Writing works out and consolidates ideas; polishing documented arguments can lower a barrier unrelated to scientific quality, since AI writing assistance is used disproportionately by researchers from non-English-speaking countries.Source: §3.6 · pp. 10–11
  • Fields where argument and prose cannot be separated, such as philosophy, may reasonably restrict AI as a writing assistant.The recommendation to use AI for elaboration or polishing applies after authors have worked out and documented their argument.Source: §3.6 · pp. 10–11
  • Before submission, AI feedback can catch clear errors and check venue conventions, leaving human reviewers to focus on substance and understanding the work.Source: §3.6 · pp. 10–11

3.7 Community evaluation

Peer review and post-publication critique should remain human, while independent replication can be delegated. Human evaluation tests both whether others can understand a contribution and whether it matters to its community.

3.7 Community evaluation

  • Peer review and post-publication critique should remain human because they test whether others understand the work and whether it matters to the community it addresses.Source: §3.7 · p. 11
  • Independent replication can be delegated, especially when large-scale replication would otherwise exhaust human effort, but hand replication remains valuable training for early-career researchers.Replication is closer to validation because it confirms that results reproduce.Source: §3.7 · p. 11

3.8 Supervision

Supervision spans research phases and sustains durable scientific understanding through skill-building and mentorship. AI reliance can expand as researchers become competent, provided each use still passes the understanding and direction test.

3.8 Supervision

  • Early-career researchers should perform skill-building work without AI until competent, while senior researchers should learn the tools, guide their use, and receive institutional recognition for mentorship.Source: §3.8 · p. 11
  • Competence, as judged by senior community members rather than a fixed number of repetitions, marks when early-career researchers can expand AI reliance in skill-building phases.Depending on the field, supervisors may allow AI assistance in phases where delegation is permitted, such as validation and polishing.Source: §3.8 · p. 11
  • Mentorship passes scientific judgment between generations and merits recognition even when it reduces senior researchers’ publication counts.Supervision is oversight and mentorship across all research phases.Source: §3.8 · p. 11

4 Reforms for Science Built on Human Understanding and Direction

As AI makes correct papers and polished prose cheap, the paper argues that institutions must reward the people and practices that sustain human understanding and direction.

Reforms for science built on human understanding and direction

  • AI makes correct papers and polished prose cheap, so institutions must stop privileging speed, volume, and short-term productivity over understanding and mentorship.The authors argue that individual researchers cannot maintain these norms without institutional change.Source: §4 · pp. 11–12

Durability

  • Universities and funders should reward mentorship, while doctoral programs test students’ independent command of core tasks rather than police unverifiable AI use.Oral defenses, live analyses, and critiques of unfamiliar work test whether trainees have acquired the skills the community needs to retain.Source: §4 · p. 12

Collective understanding

  • Evaluations should reward selected clear, important work and synthesis; venues should retain human-only review, train reviewers to test contestability, and limit submissions.Review articles, textbooks, and teaching merit more credit; reviewers should treat claims they cannot follow as feedback, while submission caps protect scarce human review.Source: §4 · p. 12

Human direction

  • Funders should require phase-specific AI plans, support explanation, replication, and teaching of AI-derived results, and keep diverse questions open through portfolio choices and accountable tools.The authors note evidence that AI use narrows the range of ideas and topics; publicly governed or open scientific models would keep tool influence accountable to the scientific community.Source: §4 · p. 12

Transparency

  • Venues should require CRediT-phase disclosure of AI and human work plus tiered citations; automated checks can assess completeness, but accuracy remains human responsibility.The authors argue that principled use must carry no stigma and reject blanket bans, which would forgo legitimate uses and push AI use out of sight.Source: §4 · pp. 12–13

AI use statement

Heidari discloses how AI contributed to the paper, distinguishing work done without AI from literature review, feedback, drafting, and revision. The account also records independent and expert review and the author’s responses to them.

AI use statement

  • Heidari used Claude for literature search, critique, and later drafting, and ChatGPT for an independent review.The disclosure frames the account as a response to the paper’s argument for judicious AI use in research.Source: AI use statement · p. 13, Heidari’s statement · p. 13

Heidari’s statement

  • Heidari wrote the thesis, outline, phase-by-phase recommendations, and initial draft without AI, and chose the framework’s single principle and “human direction” label.Source: Heidari’s statement · p. 13
  • Claude checked an initial list of 18 papers, added about 120 works with synopses, and critiqued the scope and argument; Heidari verified citations and chose what to read.Claude suggested building on a single principle and using CRediT; Heidari adopted its suggested wording for the definition and test with minor changes.Source: Heidari’s statement · p. 13
  • Claude drafted later revisions and specified sections from Heidari’s supplied arguments, while Heidari edited those drafts and revised the manuscript after ChatGPT’s independent review.Claude also copy-edited Revision 5, revised the abstract and contribution summary, and made consistency edits for Revision 7.Source: Heidari’s statement · p. 13
  • Colleague questions and a philosopher of science’s written feedback informed later revisions; Claude drafted wording for changes that Heidari then edited.The account specifies that Claude drafted several passages from ideas and points supplied by Heidari and adapted wording proposed by the expert.Source: Heidari’s statement · pp. 13–14
Loading 2610.36116v1…