No Hidden Prompts Needed! You Can Game AI Peer Review with Presentation-Only Revisions

B Project Website: https://xyimatvoid.github.io/ARGAR-Site/

• The disclaimer jokes that any resemblance to a clearer, better-framed version is coincidental.It begins, “We promise this paper has not been adversarially repackaged.”Source: B Project Website:… · p. 1

1. Introduction

• Adversarial repackaging uses reviewer feedback to iteratively search presentation-level edits while holding methods, experiments, figures, equations, and numerical results fixed.The targeted content includes abstracts, contribution statements, related work, discussion, and narrative structure; these visible edits fall within normal academic writing practices.Source: §1 · pp. 2–3
• The authors argue that rewarding presentation optimization over genuine contribution improvement can distort peer-review incentives, and report that the vulnerability spans multiple mainstream reviewer models and review templates.They frame this as a structural deficiency of current AI reviewers rather than a defect of one model.Source: §1 · pp. 2–3
• 75.1% attack success rate and +1.21 mean score gain show that adversarial repackaging improves scores across three mainstream AI reviewers and different review templates.The attack iteratively optimizes presentation-level edits while keeping scientific content fixed.Source: §1 · pp. 2–3
• The paper identifies a strength–weakness asymmetry and a strategy effectiveness gradient: highlighting strengths is easier than rebutting criticisms, and presentation strategies differ in effectiveness.Attempts to rebut criticisms may backfire; the variation in strategy effectiveness indicates a systematic vulnerability.Source: §1 · p. 3
• The paper constructs a contamination-free rolling dataset of recent unpublished arXiv preprints with LaTeX sources and PDFs, using automatic multi-stage filtering to support realistic AI-assisted review evaluation.The dataset aims to provide representative coverage while reducing test-set contamination.Source: §1 · p. 3

AI review systems and evaluation

• Prior AI-review studies document score inflation, convergent focus, low agreement, and limited perspective diversity, but do not systematically test manipulation through presentation-level edits.The cited studies characterize review quality limitations rather than whether presentation changes can manipulate scores.Source: §2 · p. 3

Prompt injection and hidden-text attacks

• Hidden-text prompt-injection attacks [29] [32] [33] are prohibited by most conferences and expose patchable vulnerabilities rather than structural deficiencies in AI-review evaluation.These attacks embed invisible instructions in papers to manipulate review outputs.Source: §2 · p. 3

Surface-level textual perturbation

• Lin et al. [17] use non-semantic NLP perturbations, including synonym substitution and style transfer, to target attended regions and inflate scores without deeply analyzing failure mechanisms.The passage also names Jin et al. 9 in connection with these conventional attacks.Source: §2 · p. 3

Paper rewriting and laundering

• Kaneko [10] optimizes abstract paraphrases with review-score feedback but uses only scalar scores and over two thousand API calls per paper; Baumann et al. [3] rewrite full papers without separating science from presentation.The full-paper laundering approach achieves limited score gains, and both methods demonstrate feasibility without analyzing underlying success mechanisms [8].Source: §2 · pp. 3–4
• Adversarial repackaging achieves 75.1% ASR and a +1.21 mean score gain, while revealing strength-weakness asymmetry and strategy-dependent effectiveness.Unlike prior rewriting approaches, it enforces presentation-level constraints that hold scientific content fixed and uses closed-loop iterative optimization.Source: §2 · p. 4

3.1. Threat Model

• The threat model permits framing and narrative changes, limits technical exposition to factual rephrasing or reorganization, and keeps data, tables, figures, equations, proofs, and numerical results fixed.The attacker edits LaTeX and queries the reviewer as a black box, without access to its prompts or parameters.Source: §3.1 · p. 4

3.2. Attack System

• The attack repeatedly profiles reviews, plans and compiles edits, evaluates candidate reviews, and retains a best-so-far version rather than accumulating every prior change.A persistent history records signals, strategies, plans, and outcomes, allowing recovery from failed modifications.Source: §3.2 · pp. 4–5
• Edits are selected from reviewer-specific signals: high-severity unresolved issues receive responses, backfiring directions are avoided, and recognized strengths are protected.The strategy pool contains 20+ presentation-level approaches, including narrative restructuring and surface editing, without changing scientific content.Source: §3.2 · p. 5

3.3. Evaluation Protocol

• Candidate revisions must pass a direction gate on perceived strengths, weakness severity, and net content improvement, then outperform the current best on a composite score.Pairwise comparisons against fixed baseline reviews assess strength and weakness changes; the two-stage rule rejects score gains that worsen criticism or weaken recognized strengths.Source: §3.3 · pp. 5–6
• Final evaluation compares attacked and baseline reviews on content changes and reports mean score shift and attack success rate, defined as the proportion of papers with ΔS≥+1.The protocol reports Δstrength, Δseverity, and Δcontent alongside the two numerical metrics.Source: §3.3 · p. 6

Dataset

• The benchmark contains over 500 unpublished ML, CV, and NLP preprints, with a reproducible construction pipeline currently covering submissions through April 2026.Papers are paired LaTeX sources and compiled PDFs, filtered to genuine research submissions with moderate review scores.Source: §4 · p. 6

Reviewer models and review generation

• Three reviewer models—Claude Sonnet 4, Claude Sonnet 4.5, and GPT-5-mini—review compiled PDFs using complete ICLR, NeurIPS, and ICML guidelines.Each paper receives N independent reviews; the default template uses ICLR guidelines.Source: §4 · p. 6

Attack configuration

• Main attack campaigns match the attacker model to the target reviewer, run multiple rounds of the six-stage loop, and use a separate model as pairwise judge.Cross-model transferability analyses are provided in Appendix F.Source: §4 · p. 6

Baselines

• Baselines isolate missing design elements: Zero-shot Paper Laundering [3] omits iterative optimization, PAA [10] limits edits to the abstract, and Research Agent lacks adversarial strategy selection.All three are evaluated on a subset; Research Agent performs iterative full-paper revision without adversarial objectives or strategy selection.Source: §4 · p. 6

5.1. Overall Attack Effectiveness

• A single rewrite, abstract-only optimization, and iterative full-paper revision without adversarial strategy selection each lacked a component of the proposed approach.The paper reports that zero-shot Paper Laundering [3] yielded limited improvement without closed-loop iteration, PAA [10] was limited to abstract edits, and the Research Agent lacked adversarial objectives and strategy selection.Source: §5.1 · p. 7
• ΔS +1.21 and ASR 75.1% show that presentation-only edits systematically improved scores across three AI reviewers.The edits left methods, experiments, figures, and numerical results unchanged; all full-dataset ΔS values were statistically significant (Wilcoxon signed-rank test, p<0.0001).Source: Table 1 · p. 7, §5.1 · p. 7

5.2. The Strength-Weakness Asymmetry

• AI reviewers were more readily impressed by amplified strengths than convinced weaknesses were resolved: strengths improved in 86.1% of rounds (mean +2.19), while weakness severity increased in 31.6%.The 31.6% weakness backfire rate was 2.6 times the 12.4% strength backfire rate.Source: §5.2 · p. 8

5.3. Strategy Effectiveness Gradient

• Related-work repositioning and analytical discussion expansion sustained higher accepted exposure than surface edits, while contribution-list enhancement most often appeared at first breakthroughs.First-hit rates were 87.2% for contribution-list enhancement, 66.0% for analytical discussion expansion, and 44.7% each for related-work repositioning and self-deprecation removal. Accepted exposure was 49.3% for related-work repositioning and 44.9% for analytical discussion expansion, versus a 30.8% baseline; surface strategies included table formatting (29.8%), local polishing (27.8%), and algorithm boxes (26.5%). Analytical discussion expansion had the largest effects: Δstrength +2.63 and Δseverity −1.98.Source: §5.3 · p. 9, Table 2 · p. 9

5.4. Case Study: Anatomy of a Review Manipulation

• In a case study, presentation edits raised a paper’s review from 3/10 (Reject) to 6/10 (Weak Accept) without changing its scientific content.The paper’s same two datasets, five methods, and numerical results remained unchanged. Reviewers echoed strengthened self-positioning, while reframed and decoy limitations displaced original weaknesses; the case illustrates reviewers treating the appearance of addressing an issue as if it resolved it.Source: §5.4 · p. 10

6. Conclusion

• The paper proposes resistance to presentation-only review gaming as a necessary condition for AI review automation.Source: §6 · p. 10
• With scientific content entirely fixed, presentation-level edits raise review scores across multiple mainstream models and review templates.Reframing how reviewers understand a paper is more effective than surface improvements, while attempts to dissolve specific criticisms frequently backfire.Source: §6 · p. 10
• Satisfying this condition alone would not justify safe deployment; the authors release an attack framework and contamination-free dataset for adversarial evaluation.The paper directs readers to Appendix I for alternative explanations, broader implications, and deployment recommendations.Source: §6 · p. 10

• Experiments cover three Claude- and GPT-series reviewer configurations, with additional models untested because of compute constraints.Source: Limitations · p. 11
• Consistent vulnerability across configurations and positive cross-model transfer suggest a structural property of current AI reviewers, though testing more model families would strengthen this conclusion.Source: Limitations · p. 11
• For papers with concrete experimental gaps, presentation optimization gains plateau around 5.0–5.5, mostly within the first few rounds, indicating retained sensitivity to substantive shortcomings.Source: Limitations · p. 11

Ethical Considerations

• The authors disclose the dual-use vulnerability to support robustness testing, and release a reproducible dataset pipeline and protocol rather than venue-targeting attack tools.The pipeline uses publicly available arXiv preprints and no private or personally identifiable data; the authors argue nondisclosure risks deployment of insufficiently validated review systems.Source: Ethical Considerations · p. 11

A. Presentation-Level Strategy Pool

• The strategy pool separates narrative restructuring, which changes interpretation of contributions, positioning, and limitations, from surface editing, which improves presentation without changing framing.Examples include related-work repositioning, analytical discussion expansion, abstract reframing, and contribution-list strengthening; all strategies leave methods, experiments, figures, equations, and numerical results unchanged.Source: A. Presentation-Level Strategy Pool · p. 16, Table 3 · p. 17, §B.1 · p. 17

B. Dataset Construction

• The benchmark pairs LaTeX sources with compiled PDFs to capture the source-editing and PDF-review workflow that static, text-only benchmarks miss.Prior benchmarks often use published papers, risking pretraining contamination, and overlook PDF-mediated effects from layout, figures, tables, equations, and appendix references.Source: B. Dataset Construction · p. 16

B.1. Construction pipeline

• The automated collection retrieves arXiv preprints posted through April 2026 across machine learning, computer vision, and natural language processing, making the benchmark rolling by design.It downloads both editable LaTeX sources and compiled PDFs and can be rerun to incorporate new preprints.Source: §B.1 · p. 16
• The filtering pipeline retains recent, substantive papers: deduplication and quality screening retain about 84%, while dual publication-status verification retains about 67%.Papers must be 12–35 pages and have at least two authors; verification combines arXiv metadata and Semantic Scholar records to reduce contamination from published work.Source: §B.1 · pp. 16, 18
• Source and review checks retain about 29% after compilation verification and about 19% after multi-model cross-review.Compilation requires a working LaTeX source and PDF; multiple frontier models apply ICLR, NeurIPS, and ICML guidelines to screen for genuine research, sound methods, no fatal flaws, and moderate review scores.Source: §B.1 · pp. 18–19
• The dataset spans 11 research areas; retained papers have a median of 18 pages and 4 authors, with Deep Learning at 33% and ML for Sciences at 17%.Page counts range from 12–35 (mean 18.5), and author counts from 2–20 (mean 4.6).Source: Figure 3 · p. 18, §B.1 · p. 19

C. System Architecture Details

• Figure 4 presents the system architecture’s three-layer design and information flow across attack rounds.The supplied passage identifies Figure 4 as a detailed view of the architecture outlined in §3.2 but does not describe the layers further.Source: C. System Architecture Details · p. 19

D. Experimental Details

• The reviewers were claude-sonnet-4-2025, claude-sonnet-4-5-20250929, and gpt-5-mini; Claude temperature was 0.1, while GPT-5-mini used API-default 1.0.Reviews used compiled PDFs submitted through multimodal APIs.Source: D. Experimental Details · p. 19
• Review prompts combined system instructions, venue guidelines, and the paper PDF; each round averaged 3 independent reviews with structured scores and textual feedback.ICLR 2025 guidelines were the default, with NeurIPS and ICML guidelines substituted in cross-template transferability experiments.Source: D. Experimental Details · pp. 19–20
• Matched attack agents used the reviewer checkpoint, ran up to 8 rounds, and required strength gain ≥1.0, weakness worsening ≤1.0, and net benefit >0.8; selection weights were 0.8 and 0.2.Claude attack-agent temperature was 0.9; GPT-5-mini remained at API-default 1.0.Source: D. Experimental Details · p. 20
• The pairwise judge was claude-sonnet-4-5-20250929 at temperature 0.0 and synthesized review-pair judgments into a dominant trend rather than an arithmetic mean.For each pair, it reported strength_delta and severity_delta on −10 to +10 scales.Source: D. Experimental Details · pp. 20–21
• Zero-shot Paper Laundering [3] used one feedback-informed full-paper rewrite; PAA [10] ran 8 rounds of 3 abstract candidates, while Research Agent used fixed strategies without adaptive adversarial selection or a direction gate.Research Agent matched the attack system's base LLM, number of rounds, and reviewer queries.Source: D. Experimental Details · p. 21

Score calibration

• AI reviewer scores were significantly higher for accepted papers (p<.005), correlated moderately with human means (Pearson r=.58–.61; Spearman ρ=.56–.58), and calibrated less well with more mid-scoring papers.The weaker calibration does not affect the study’s focus on directional score changes rather than absolute calibration.Source: Table 4 · p. 21, E. Robustness Validation of LLM-Based… · p. 21

Test-retest reliability

• The attack’s cross-model mean score gain of +1.21 far exceeded the small within-paper score variation across three independent reviews.The reported standard deviation is measured on the ICLR 1–10 rating scale.Source: E. Robustness Validation of LLM-Based… · p. 22, Table 5 · p. 22

Content calibration

• Across ICLR 2024 and 2025 review pairs, the pairwise judge’s directional accuracy was high and improved monotonically with larger human rating gaps.The nested evaluation used 64 pairs at gap ≥5, 80 at gap ≥4, and 96 at gap ≥3; it tested severity and strength directions using review text alone.Source: Table 6 · p. 22, E. Robustness Validation of LLM-Based… · p. 22

Order invariance

• On 96 review pairs with rating gap ≥3, directional consistency under swapped input order was 100% for Δseverity and ≥98% for Δstrength.The judge was run on each pair in its original and reversed order.Source: Table 7 · p. 22, E. Robustness Validation of LLM-Based… · pp. 22–23

F. Transferability

• Transfer tests reused attacked papers and required only re-evaluation under different reviewer models or templates, without rerunning the attack.Cross-model evaluation used three independent reviews per model pair.Source: F. Transferability · p. 23
• Mean ΔS was +1.42 in the matched setting and +0.88 in the mismatched setting; every off-diagonal model-transfer result was positive, including the weakest cross-family pair.Transfer was stronger within the Sonnet 4/Sonnet 4.5 family than across Claude and GPT.Source: Table 8 · p. 23, F. Transferability · p. 23
• ICLR-optimized attacks produced positive score gains under both NeurIPS and ICML guidelines, though ΔS values are not directly comparable across their different rating scales.The result indicates that presentation-level improvements did not overfit ICLR-specific evaluation criteria.Source: Table 9 · p. 23, F. Transferability · pp. 23–24

G.1. Blind pairwise semantic-preservation audit.

• Annotators compared randomized Version A and Version B without knowing which was original, rating five scientific-content dimensions on a 0–2 preservation scale.The dimensions were core contribution, method or technical approach, experimental setup, reported results or evidence, and conclusions or main claims.Source: §G.1 · p. 24, Table 10 · p. 25
• Pairs were counted as preserved when τ=0.8, corresponding to an average score of at least 8 out of 10 across annotators and dimensions; annotator-level preservation scores were averaged equally.Inter-annotator agreement was analyzed separately using annotator-specific preserved/not-preserved labels before aggregation.Source: §G.1 · pp. 24–25
• 20/30 blinded pairs met the τ=0.8 preservation threshold (66.7%); the mean pairwise preservation score was 0.80/1.00, with 83.3% agreement and Fleiss’ κ=0.62.Three annotators independently evaluated 30 pairs.Source: §G.1 · p. 25
• Experimental setup preservation scored 1.93, while Conclusions / main claims scored 0.97, the highest and lowest dimension-level scores, respectively.Method/technical approach and experimental setup were near-ceiling, while lower scores concentrated on narrative framing.Source: Table 11 · p. 25, §G.1 · p. 25

H. Case Study: Full Analysis

• 3/10 (Reject) rose to 6/10 (Weak Accept) after presentation-only edits, with all three sub-scores increasing from 2 to 3 despite unchanged scientific content.Figure 5 depicts the transformation; the study reports the same datasets, methods, tables, figures, equations, and numerical results before and after editing.Source: Figure 5 · p. 26, H. Case Study: Full Analysis · p. 26
• The case study traces four mechanisms: rationalizing an omitted comparison, reframing two datasets as complementary, using decoy limitations to steer criticism, and formalizing existing theory to gain theoretical credit.The reviewer softened or dropped criticisms despite no corresponding scientific additions; a discussion passage also recast high Local Procrustes correlation as complementary value over geometric measures.Source: H. Case Study: Full Analysis · pp. 27–28
• The reviewer upgraded all five original strengths and added one, while removing three of six weaknesses, softening three, and introducing two new ones from planted limitations.It echoed self-positioning such as “the first” and “complementary data regimes,” illustrating strength inflation alongside limitation laundering.Source: H. Case Study: Full Analysis · pp. 26–28

I.1. Alternative Explanations

• +1.21 mean score gain exceeds 0.10–0.28 within-paper standard deviation; validation also reports >90% directional accuracy, ≥98% order consistency, and transfer across models and templates.The discussion argues that improved clarity alone cannot explain reviewers’ shifts toward treating the same evidence as stronger science.Source: §I.1 · pp. 28–29

I.2. Broader Significance

• Because edits are visible and legitimate yet can be either clarification or manipulation, no simple surface check can distinguish them, while low-cost gaming risks shifting incentives toward packaging over science.The paper says this ambiguity makes resistance necessary but insufficient for safe deployment.Source: §I.2 · p. 29

I.3. Implications for Deployment

• The authors recommend testing reviewers with presentation-only perturbations and withholding acceptance influence from systems not anchored to scientific content, while scoring scientific soundness separately from writing quality.They identify narrative-changing edits as more effective than surface improvements and offer their attack framework and contamination-free dataset as a benchmark.Source: §I.3 · pp. 29–30

J. Prompt Templates

• The prompt appendix specifies shared review instructions and base64 PDF transmission, a pairwise judge comparing baseline and candidate reviews with per-pair aggregation, and cross-template experiments using NeurIPS and ICML guidelines.The alternative templates differ from ICLR in scoring dimensions and scales.Source: J. Prompt Templates · p. 30, §J.2 · p. 30, §J.3 · p. 30

J.3. Cross-Template Review Prompts

• The cross-template prompt asks a meta-reviewer to synthesize matched baseline–candidate review pairs, prioritize the dominant pattern, note sharp outliers, and report overall strength and severity deltas.It requests detailed analysis followed by a JSON result containing the synthesis and two deltas.Source: §J.3 · p. 33
• The NeurIPS 2025 template requests a summary, strengths and weaknesses across quality, clarity, significance, and originality, author questions, limitations, and an overall score.It also requests confidence and a single JSON review with the specified fields.Source: §J.3 · p. 34
• The ICML 2026 template uses soundness, presentation, significance, and originality criteria and requests a summary, author questions, limitations, overall score, confidence, and JSON output.Its review format also includes strengths and weaknesses and an ethics flag.Source: §J.3 · p. 35