Source-linked AI summary

Position: Academic Conferences are Potentially Facing Denominator Gaming Caused by Fully Automated Scientific Agents

Rong Shan, Te Gao, Hang Zheng, Yunjia Xi, Jiachen Zhu, Zeyu Zheng, Yong Yu, Weinan Zhang, Jianghao Lin

arXiv:2605.09915v1cs.CLcs.AIcs.CY

TL;DR

The paper examines how stable acceptance-rate policies combined with rapidly advancing scientific agents create a vulnerability to Agentic Denominator Gaming. It formalizes the attack, assesses its economic and operational feasibility, and evaluates technical and policy defenses. It concludes that durable protection requires system-level reforms because detection alone is vulnerable to adversarial evasion and can impose new burdens and biases.

  • Problem

    Stable acceptance rates allow malicious actors to inflate submission denominators with low-quality AI papers, potentially increasing targeted papers’ acceptance odds while burdening peer review.

    Source: §1 · pp. 1–2, §2.1 · p. 3
  • Method

    The paper formalizes the threat, analyzes its economics and review-system consequences, and compares technical, policy-based, and systemic mitigation strategies.

    Source: §5.1 · p. 7, §5.2 · p. 7, §7 · p. 9
  • Results

    GPT-4o completed the paper-generation task in 1,165.4 s for $2.33 with a 94.3% success rate, while the submission agent cost less than one dollar.

    Source: §4.1 · p. 5
  • Takeaways & Limitations

    Academic conferences should pursue proactive system-level defenses rather than rely primarily on AI detection against determined adversaries.

    Source: §5.1 · p. 7, §7 · p. 9

    Detection-based defenses face unreliable screening, adversarial evasion, false positives, and burdens that can disadvantage legitimate authors.

    Source: §5.1 · p. 7, §6.1 · p. 8, §6.2 · p. 9

Abstract

from arXiv · show

The implicit policy of maintaining relatively stable acceptance rates at top AI conferences, despite exponentially growing submissions, introduces a critical structural vulnerability. This position paper characterizes a new systemic threat we term Agentic Denominator Gaming, in which a malicious actor deploys AI agents to generate and submit a large volume of superficially plausible but low-quality papers. Crucially, their objective is not the acceptance of low-quality papers, but rather to inflate the submission denominator and overwhelm reviewing capacity. Under a relatively stable acceptance rate, this dilution can systematically increase the publication probability of a small, targeted set of legitimate papers. We analyze the practical feasibility of this threat and its broader consequences, including intensified reviewer burnout, degraded review quality, and the emergence of industrialized automated agent mills. Finally, we propose and evaluate a range of mitigation strategies, and argue that durable protection will require system-level policy and incentive reforms, rather than relying primarily on technical detection alone.

1. Introduction

The paper identifies Agentic Conference Denominator Gaming as a systemic threat in which automated agents inflate submission pools to manipulate stable acceptance-rate mechanisms. It motivates proactive, system-level mitigations for the resulting burdens on review quality and scientific trust.

  • Automated scientific workflows and paper-writing tools have lowered the marginal cost of producing conference-style manuscripts at scale.Source: §1 · p. 1
  • Stable acceptance rates make submission counts a controllable denominator that can shape each paper’s marginal acceptance probability.Source: §1 · pp. 1–2
  • Agentic Conference Denominator Gaming uses plausible-but-low-value submissions as strategic noise to improve targeted human-authored papers’ relative acceptance odds.The attack manipulates the publication pipeline rather than seeking acceptance for the generated papers themselves.Source: §1 · pp. 1–2
  • The threat can impose uncompensated review labor, worsen reviewer burnout and quality, and industrialize automated paper mills across venues.The paper links these effects to cascading publication-pipeline risks and erosion of scientific authenticity.Source: §1 · p. 2
  • The paper formalizes the threat and seeks proactive, system-level mitigations before automated attacks become routine.Source: §1 · p. 2

2. Agentic Conference Denominator Gaming

This section connects rapidly growing submissions and increasingly capable scientific agents to a vulnerability created by stable acceptance-rate policies. It formalizes how low-quality AI submissions can raise human acceptance odds without improving paper quality.

2.1. Submission Growth and Stable Acceptance Rates

  • NeurIPS submissions grew from about 6,700 in 2019 to over 30,000 in 2025, while venues such as CVPR, NeurIPS, and EMNLP maintained acceptance rates around 20–30%.The paper describes this stability as visible across conference scopes and reports standard-deviation analyses.Source: §2.1 · p. 3, Table 1 · p. 3
  • Stable acceptance rates are an implicit policy choice that can make acceptance thresholds function as administrative targets rather than purely quality-based filters.Source: §2.1 · p. 3

2.2. The Rise of Scientific Agents

  • Scientific-agent frameworks can autonomously perform literature review, experiments, reporting, formatting, and even peer review, making thousands of plausible papers technically and economically feasible.The AI Scientist framework is cited as estimating roughly $15 per paper.Source: §2.2 · p. 3

2.3. The Mechanism and Objective of Agentic Conference Denominator Gaming

  • Agentic Denominator Gaming floods a venue with superficially plausible, intentionally low-quality manuscripts that expand the submission denominator while remaining unlikely to be accepted.Source: §2.3 · pp. 3–4
  • The attacker’s objective is to improve acceptance odds for a smaller set of targeted manuscripts, hacking stable-rate game parameters rather than improving intrinsic quality.Source: §2.3 · pp. 3–4
  • As the AI-to-human submission ratio rises, the effective acceptance rate for unchanged human submissions rises proportionally under a stable target rate.Source: §2.3 · pp. 3–4

3. Threat Modeling

The threat model separates paper generation from submission logistics in a two-agent pipeline designed for scale and minimal human oversight. A controlled end-to-end test confirmed that the generation–registration–submission chain is executable in practice.

3. Threat Modeling

  • The proof-of-concept architecture uses a Research Agent for manuscript generation and a Submission Agent for automated conference-platform submission.The division decouples content creation from submission logistics and supports modular scaling.Source: §3 · p. 4, Figure 4 · p. 5

3.1. The Research Agent for Plausible Paper Generation

  • The Research Agent produces stylistically plausible but scientifically invalid papers and adapts them to conference templates to reduce formatting-based desk rejection.Source: §3.1 · p. 4

3.2. The Submission Agent for Automated OpenReview Flooding

  • OpenReview’s documented APIs enable automated registration, form completion, metadata collection, submission, and rebuttal handling, making the submission component highly feasible.The same infrastructure designed for legitimate submission management can therefore support scaled automated attacks.Source: §3.2 · pp. 4–5
  • A controlled end-to-end test successfully completed paper drafting, OpenReview registration, and automated submission, but was run once at minimal scale and promptly removed.Source: §3.2 · p. 5

4. Broader Impacts and Systemic Risks

The paper argues that automated denominator gaming creates systemic risks by shifting low-cost burdens onto reviewers and enabling industrialized academic fraud at unprecedented scale. These effects can degrade review quality, erode conference trust, and threaten the publication ecosystem.

4.1. The Asymmetric Economics of Attack and Defense

  • A minimal attacker investment can impose millions of dollars in collective academic opportunity costs through reviewer labor and cross-venue reuse of synthetic manuscripts.GPT-4o generation is reported at $2.33 per task, while defense requires thousands of hours of uncompensated expert labor; rejected manuscripts can be resubmitted across venues.Source: §4.1 · pp. 5–6

4.2. Exacerbating Reviewer Burnout and Eroding Scientific Trust

  • Agentic denominator gaming would intensify reviewer burnout by flooding review pools with low-quality papers, increasing cognitive waste and driving experienced reviewers away.The resulting brain drain would leave legitimate authors with less feedback from an increasingly overwhelmed and less qualified reviewer pool.Source: §4.2 · p. 6
  • Declining review quality and manipulated acceptance thresholds could erode conference prestige, causing strong researchers to redirect submissions and accelerating further decline.The paper describes this as a vicious cycle that undermines trust in conference publication as a signal of achievement.Source: §4.2 · p. 6

4.3. From Paper Mills to Agent-driven Paper Mills: The Automation of Academic Fraud

  • Agent-driven paper mills replace labor-intensive fraud with coordinated AI systems that can generate and submit fraudulent papers at unprecedented scale.The paper characterizes this shift as an assembly-line operation rather than a cottage industry.Source: §4.3 · p. 6
  • Agent-driven mills could generate hundreds or thousands of papers daily, collapse production costs, and make industrial-scale academic fraud accessible to diverse malicious actors.The resulting volume could overwhelm submission systems and finite human review capacity.Source: §4.3 · p. 6
  • By producing unique text, fabricated data, plausible figures, self-citation webs, and adaptive tactics, agent-driven mills could evade existing detection and create an epistemic-security crisis.Published fraudulent papers could then be indexed and used to train later language models, polluting the information ecosystem at its source.Source: §4.3 · pp. 6–7

5. Potential Defenses Strategies

The paper proposes layered technical, policy-based, and systemic defenses against Agentic Conference Denominator Gaming, while arguing that technical detection alone is unreliable and unsustainable. The strongest structural remedies alter submission costs, review stages, or the link between submission volume and acceptance outcomes.

5.1. Technical Defenses

  • Technical defenses combine automated triage with similarity, citation, artifact, and metadata checks, while separately raising the cost of automated account creation and mass submission.The proposed measures target both malicious scientific agents and malicious submission agents, using multiple signals and identity or rate controls rather than a single detector.Source: §5.1 · p. 7
  • Detection tools remain unreliable because false positives create serious reputational and legal risks, while adversarial paraphrasing can reduce detection rates by over 98%.The paper therefore characterizes detector-based defenses as an unsustainable cat-and-mouse strategy against evolving generation and evasion techniques.Source: §5.1 · p. 7

5.2. Policy-Based Defenses

  • Submission fees could make high-volume flooding financially prohibitive, reduce reviewer burden, and fund reviewer rewards, but they risk excluding under-funded legitimate researchers.The paper presents fees as a commitment mechanism that may improve submission quality while creating significant equity concerns.Source: §5.2 · pp. 7–8
  • Multi-stage review would reject fundamentally flawed submissions early and reserve intensive discussion and rebuttal for a smaller pre-vetted cohort.This triage structure is intended to concentrate scarce reviewer attention on promising manuscripts and improve resilience to low-quality submission floods.Source: §5.2 · p. 8

5.3. Systemic Defenses

  • A bounded acceptance-number range would sever the link between submission volume and acceptances, directly neutralizing denominator inflation, but could force rejection of strong papers.The reform also requires governance and cultural changes because conferences would move away from acceptance rate as a central selectivity metric.Source: §5.3 · p. 8
  • Author reputation systems could hinder anonymous, Sybil-like scaling by requiring endorsements and penalizing confirmed fraudulent or AI-generated submissions.The proposed system builds reputation from submission and review history, creating stronger disincentives for established researchers to attach their names to questionable work.Source: §5.3 · p. 8

6. Alternative Views

The paper argues that neither AI detection nor pre-screening adequately addresses denominator gaming because both defenses remain vulnerable to adversarial adaptation and impose substantial administrative costs. Pre-screening additionally risks procedural bias and shifts the burden onto conference leadership.

6.1. AI-Generated Papers are Easily Detectable

  • AI detectors are unreliable for high-stakes screening and can be evaded as generators adapt to their learned patterns.The paper characterizes this as an adversarial arms race in which purely technical defenses remain reactive and fragile.Source: §6.1 · p. 8

6.2. We Can Simply Defense by Implementing a Pre-Screening Phase to Filter AI Papers

  • Pre-screening shifts resource demands from reviewers to chairs, who must manually triage thousands of flagged submissions.The burden can paralyze conference executive capacity even before formal peer review begins.Source: §6.2 · p. 9
  • Rapid triage heuristics risk false positives against legitimate authors, including non-native English speakers using AI for translation or stylistic polishing.The paper distinguishes these uses from AI-generated manuscripts and warns that desk rejection can be discriminatory and demoralizing.Source: §6.2 · p. 9
  • Static screening criteria create a predictable optimization target that attackers can fine-tune agents to bypass.This makes pre-screening a temporary hurdle rather than a durable solution.Source: §6.2 · p. 9

7. Conclusion

The paper concludes that stable acceptance rates and strained volunteer review expose conferences to Agentic Conference Denominator Gaming. It argues that the threat could worsen reviewer burnout, erode prestige, and industrialize academic fraud, requiring proactive systemic defenses.

7. Conclusion

  • Stable acceptance rates and strained volunteer review create a systemic vulnerability that automated agents can exploit by flooding submission pools.The paper frames this as an economically asymmetric attack on conference selection outcomes.Source: §7 · p. 9
  • The consequences include accelerating reviewer burnout, eroding scientific prestige, and industrializing academic fraud through agent-driven paper mills.Source: §7 · p. 9
  • Reactive technical fixes such as AI detection are insufficient against determined adversaries, so protection should combine political and systemic strategies.The conclusion calls for resilient policies before the vulnerability is widely exploited.Source: §7 · p. 9

A.1. Attack Incentives

The paper argues that attackers may rationally pursue denominator gaming despite incidental benefits to other authors because they can target specific subfields and weaponize synthetic identities. Shared-metric manipulation can therefore remain attractive when private gains outweigh concern for collateral effects.

A.1. Attack Incentives

  • Subfield targeting can concentrate acceptance-threshold effects within the attacker’s niche while limiting benefits to unrelated papers.Source: §A.1 · p. 12
  • Synthetic accounts could infiltrate reviewing and deliberately suppress competing human-authored papers.Source: §A.1 · p. 12
  • The paper compares this incentive structure with ticket-market and social-media manipulation, where attackers bear costs for direct gains despite spillover benefits to others.These examples are presented as evidence that collateral benefits need not deter manipulation of shared statistical metrics.Source: §A.1 · p. 12
  • Attackers may accept incidental benefits to other authors because manipulating a shared metric can still produce private strategic gains.The paper treats this incentive as a structural and social-psychological feature rather than a deterrent.Source: §A.1 · pp. 12–13
  • The paper does not claim attacks are inevitable; it aims to establish technical feasibility and damaging effects to motivate early preventive action.Source: §A.1 · p. 13

A.2. Emerging Signs of The Attack

An analysis of NeurIPS and ICLR submissions finds rising AI-detection scores over time, especially among non-accepted papers. The accompanying rating trends indicate that submission pools are expanding with a larger fraction of lower-quality papers, though the evidence is based on detection-model scores and observational comparisons.

A.2. Emerging Signs of The Attack

  • AI-detection scores increased over time for both accepted and non-accepted NeurIPS and ICLR submissions, indicating growing AI involvement.The analysis scores abstracts with ai-text-detector-v1.01, where higher scores correlate with increased probability of AI involvement.Source: §A.2 · p. 13
  • Non-accepted papers typically had higher average AI-detection scores than accepted papers, linking stronger AI signals with lower-quality submissions.Source: §A.2 · p. 13
  • The Average Annual Growth Rate rose more quickly for non-accepted than accepted papers, suggesting faster growth of AI-generated content among rejected or withdrawn submissions.Source: §A.2 · p. 13
  • Accepted-paper ratings declined from 6.59 to 6.44 to 6.43, while rejected-paper ratings fell from 4.90 to 4.94 to 4.53 between 2023 and 2025.The full submission-pool rating declined from 5.44 to 5.41 to 5.14.Source: §A.2 · p. 13
  • The authors interpret the rating pattern as evidence that the submission pool is expanding with a larger fraction of lower-quality papers.Source: §A.2 · p. 13

A.3. Educational Email Plausibility

The paper withholds specific mass-email generation methods for ethical reasons while citing related work that supports the feasibility of massively acquiring educational accounts.

  • Specific mass-email generation methods are intentionally omitted because they are sensitive and could encourage misuse.Source: §A.3 · p. 13
  • Prior work documents fraudulent reviewer profiles created through university email aliases.Source: §A.3 · p. 13
  • Related studies describe automated account-registration bots and security weaknesses that enable large-scale account creation or unauthorized access.Source: §A.3 · p. 13
Loading 2605.09915v1…