Source-linked AI summary
GPT4 is Slightly Helpful for Peer-Review Assistance: A Pilot Study
Zachary Robertson
TL;DR
Growing research output strains human reviewers’ capacity, motivating interest in ways to scale peer review. In a controlled pilot with 10 participants, researchers compared helpfulness of human and GPT reviews on a predetermined scale, and separately tested model robustness to inserted-error adversarial attacks. Human and GPT reviews received comparable average helpfulness ratings, but GPT review ratings showed greater variance.
Problem
Growing research output strains human reviewers’ capacity, motivating interest in ways to scale peer review.
Source: §1 · p. 1Method
In a controlled pilot with 10 participants, researchers compared helpfulness of human and GPT reviews on a predetermined scale, and separately tested model robustness to inserted-error adversarial attacks.
Source: §1 · p. 1Results
Human and GPT reviews received comparable average helpfulness ratings, but GPT review ratings showed greater variance.
Source: §3.1 · p. 3, §5 · p. 4Takeaways & Limitations
GPT-generated reviews could help an area chair balance human-sourced reviews, even if not used directly in decision-making.
Source: §5 · p. 4The small sample, designed around usefulness to paper authors, limits generalization to area chairs or the general public and was not designed to assess peer-review usefulness broadly.
Source: §5 · p. 5
Abstract
from arXiv · showhide
In this pilot study, we investigate the use of GPT4 to assist in the peer-review process. Our key hypothesis was that GPT-generated reviews could achieve comparable helpfulness to human reviewers. By comparing reviews generated by both human reviewers and GPT models for academic papers submitted to a major machine learning conference, we provide initial evidence that artificial intelligence can contribute effectively to the peer-review process. We also perform robustness experiments with inserted errors to understand which parts of the paper the model tends to focus on. Our findings open new avenues for leveraging machine learning tools to address resource constraints in peer review. The results also shed light on potential enhancements to the review process and lay the groundwork for further research on scaling oversight in a domain where human-feedback is increasingly a scarce resource.
1 Introduction
Growing research output strains human reviewers’ capacity, motivating investigation of AI assistance. This pilot compares GPT- and human-generated reviews and tests GPT’s response to inserted errors.
1 Introduction
- Growing research output strains human reviewers’ capacities, creating demand for solutions that can scale review volume.Peer review is described as supporting the quality and integrity of scholarly publications [Burnham, 1990, Spier, 2002].Source: §1 · p. 1
- The study tests whether GPT-generated reviews can be as helpful as human reviews using a controlled comparison with 10 participants.Helpfulness was assessed on a predetermined scale.Source: §1 · p. 1
- The pilot also evaluates robustness to adversarial attacks that insert errors into papers.The authors acknowledge potential biases in the setup and frame the work as exploring practical viability where human resources may be constrained.Source: §1 · p. 1
2 Methods
The study combines human reviews and GPT-generated reviews, then assesses their format and helpfulness. GPT’s review generation uses organized notes and iterative formatting checks.
2.1 Participant Selection and Guidelines
- Ten participants reviewed one another’s authored papers using Neu-style guidelines; one participant declined GPT review of their paper.Participants were contacted by email and gave consent before GPT reviewed papers.Source: §2.1 · p. 2
2.2 GPT Review Generation
- GPT-4 generates reviews in three stages: section notes, organized notes, then a guideline-based review.The study used GPT-4 with an 8k context size.Source: §2.2 · p. 2
- An iterative check defines proper formatting as covering the ten points in the NeurIPS reviewer guidelines.The generation loop repeats until formatting is correct and records the number of reattempts.Source: §2.2 · p. 2
2.3 Review Formatting and Corrections
- Human reviews that failed formatting checks were returned to participants for correction, and all reviews were stored in a worksheet.Source: §2.3 · p. 3
3 Results
Human and GPT reviews received similar average helpfulness ratings, though GPT ratings varied more. Participants also described differences in review specificity and usefulness across review stages.
3.1 Statistical Results
- 3 was the mean helpfulness rating for both human and GPT-generated reviews on the 1-to-5 scale.GPT ratings were 3 ± 0.96, compared with human ratings of 3.1 ± 0.57; GPT’s mean formatting reattempts were 4.8 ± 1.1.Source: §3.1 · p. 3
3.2 Observations and Participant Feedback
- Human reviews tended to focus on paper-specific details, while GPT reviews consistently summarized the paper; four of nine human reviews lacked a summary.Source: §3.2 · p. 3
- Participants said GPT’s broad assessment could be more useful for fresh papers than for papers already through several review rounds.Source: §3.2 · p. 3
4 Adversarial Robustness Evaluation
The robustness experiments altered abstracts or inserted informal sentences in papers, testing whether GPT-4 identified the changes. GPT-4 more consistently raised concerns about rewritten abstracts than sentence-level errors, and performed better with a smaller context size.
4 Adversarial Robustness Evaluation
- Table 1 presents example sentences before and after the transformations used in the experiments.Source: Table 1 · p. 4
- The abstract-change test rewrote an abstract to negate a key claim, while the sentence-level test made a randomly selected sentence informal.Source: §4 · p. 4
- GPT-4 consistently raised concerns about rewritten abstracts but struggled more to identify errors inserted at the sentence level.The reported comparison covers abstract rewrites and informal sentence insertions.Source: §4 · p. 4, Table 2 · p. 4
- Across experiments using 20 papers submitted to NeurIPS and GPT-4 variants with 4k- and 32k-token context sizes, GPT-4 showed stronger performance with the smaller context size.Source: §4 · pp. 3–4
5 Discussion
GPT-generated reviews received comparable average helpfulness ratings to human reviews, but their ratings were more variable. The authors describe generally good, relevant reviews alongside omissions and limits on what the pilot can establish.
5 Discussion
- GPT and human reviews received comparable average helpfulness ratings, but GPT ratings showed greater variance, suggesting possible inconsistency.The authors suggest a few-shot approach may address that inconsistency; they also note GPT can generate multiple reviews from a consistent source for area chairs to balance human reviews.Source: §5 · p. 4
- The pilot’s small sample and author-focused design limit generalization to area chairs, the public, and peer review more broadly.The study was not designed to evaluate GPT’s usefulness for peer review more broadly.Source: §5 · p. 5
- GPT reviews commonly address contributions, strengths, weaknesses, accuracy, clarity, prior work, innovation, practical implications, relevance, and method correctness.The study found no discernible pattern in the most or least positive reviews, likely because of the limited sample size.Source: §5 · p. 5
- Reviews were generally well-structured, relevant, and helpful, but sometimes omitted requested details, gave unclear summaries, or failed to identify gaps and weaknesses.Some reviews missed the instructed Errors section or offered suggestions without specifying what to revise.Source: §5 · p. 5
- The authors recommend improving consistency in emphasizing limitations, failures, and negative results, and studying varied or multiple inserted errors.They also suggest testing errors that could significantly alter a paper’s interpretation.Source: §5 · p. 5
6 Related Work
The related work places AI review generation among efforts to scale peer review while recognizing continued arguments for human judgment. It contrasts this study’s human-evaluated end-to-end reviews with work on prediction and mechanisms for improving review.
6.1 Human Involvement in Peer-Review
- Prior work argues that humans remain necessary to judge novelty, relevance, and potential impact; this study presents end-to-end AI review generation with human evaluation as its contribution.The authors distinguish concurrent work by Liu and Shah [2023], which did not focus on full review generation or conduct a human study.Source: §6.1 · p. 5
6.2 Predicting Paper Acceptance or Rejection
- AI-based acceptance prediction is a parallel approach, while structured reviews may be more interpretable through their reasoning and verifiable claims.The passage cites prediction work by Bharti et al. [2021] and Bao et al. [2021].Source: §6.2 · p. 5
6.3 Improving Peer-Review
- Mechanism-design work addresses faithful reporting of paper quality, including Su’s [2021] self-reported paper ranking, shown optimal under certain conditions.The cited work frames self-regulatory mechanisms as a route toward more accurate, transparent, and unbiased evaluations.Source: §6.3 · p. 6
- Peer prediction and later elicitation mechanisms address how to obtain informative or truthful reports when objective verification or ground truth is unavailable.The passage discusses Miller et al. [2005], Waggoner and Chen [2013], Zhang and Chen [2014], and Kong and Schoenebeck [2019].Source: §6.3 · p. 6
- The authors situate AI review assistance within broader efforts to scale peer review while retaining human involvement and improving existing mechanisms.They also note the potential of predictive models.Source: §6.3 · p. 6
7 Ethics and Societal Impact
The authors discuss possible effects of AI-assisted review on academic norms and mentorship, alongside concerns about bias, accountability, and over-reliance. They call for research to ensure alignment with academic research values and objectives.
7 Ethics and Societal Impact
- Further prompting and fine-tuning might allow AI-assisted peer review to surpass the median conference reviewer, but this is presented as a possibility rather than an established result.The authors raise questions about effects on volunteer reviewing and say models should align with academic research values and objectives.Source: §7 · p. 6
- Automating parts of peer review may raise concerns about devaluing mentorship and losing personalized guidance that relies on human judgment, empathy, and experience.Source: §7 · p. 6
- The authors flag risks of unfair topic, methodology, or author bias, unclear accountability for wrongful decisions, and over-reliance that may stifle creativity, originality, and diversity.Source: §7 · p. 6
8 Conclusion
The authors characterize their findings as preliminary and call for larger-sample studies and fine-tuned models to assess AI tools’ potential in peer review more conclusively.
8 Conclusion
- Larger samples and fine-tuned models could help establish more conclusively whether AI tools can support peer review.The authors also identify improving GPT’s low-level paper details as a possible direction for future research.Source: §8 · p. 6
A Prompts for Review Generation
The prompts guide GPT through collecting paper-specific notes, organizing them, and producing a detailed, constructive review. They direct reviewers to assess contributions, strengths, weaknesses, correctness, clarity, and relation to prior work.
A Prompts for Review Generation
- The review workflow has GPT collect section notes, organize them consistently, then write a review using formal guidelines.The prompts also ask for concise notes that help produce a detailed, constructive, and polite review.Source: A Prompts for Review Generation · pp. 9–10
- Review notes should cover errors and unclear reasoning, contributions, strengths, limitations, correctness, exposition, prior work, and useful reference information.The synthesis instructions ask GPT to retain concrete references for errors and reconcile notes that may conflict because they were made with partial understanding.Source: A Prompts for Review Generation · pp. 9–10
- The review form asks for a paper summary and contributions, strengths, weaknesses, and an assessment of correctness and methodology.The summary can help area chairs and program chairs, and can help authors identify misunderstandings to address in their response.Source: A Prompts for Review Generation · pp. 10–11
- Strengths are assessed through claim soundness, contribution significance and novelty, and relevance to the NeurIPS community.The guidance favors work that explores new territory or opens research directions over incremental advances.Source: A Prompts for Review Generation · pp. 10–11
- Reviewers should explain weaknesses along the same evaluative dimensions and make comments detailed, specific, polite, and constructive.The prompts emphasize helping authors understand the reviewer’s viewpoint rather than making vague or subjective complaints.Source: A Prompts for Review Generation · p. 11
5. Clarity: Is the paper well written?
The review guidance asks reviewers to assess clarity, prior-work context, reproducibility, and broader impact, while following calibrated scoring and feedback instructions. It also explains how to handle contemporaneous work and additional comments.
5. Clarity: Is the paper well written?
- Reviewers should identify unclear writing and give examples of passages needing revision.They should also assess whether the submission explains how it differs from prior contributions and relates its work to the literature.Source: §5 · p. 11
- Authors need not make detailed comparisons to non-refereed work or papers appearing less than two months before the deadline.The guidance treats such recent papers as contemporaneous, while encouraging comparisons for camera-ready versions of accepted papers.Source: §5 · p. 11
- Reviewers should assess reproducibility and list inadequate reproducibility among a submission’s weaknesses.The guidance asks whether enough detail is available to reproduce the major results.Source: §5 · p. 11
- Reviewers may add questions, suggestions, and other feedback in the additional-feedback section..Source: §5 · pp. 11–12
- The overall score should reflect the reviewer’s assessment of a submission’s contributions, not conference acceptance rates or sample representativeness.The rubric ranges from 10 for groundbreaking work in the top 5% of accepted NeurIPS papers to 1 for trivial, wrong, or already-known work.Source: §5 · p. 12
- Reviewers are asked to judge whether authors adequately address potential ethical and societal consequences of their work.The guidance calls for a balanced account of positive and negative outcomes.Source: §5 · p. 13
B Sample Reviews
Appendix B presents sample reviews of three papers, showing how reviews assess contributions, strengths, and limitations.
AnoFormer
- AnoFormer is presented as a transformer-based GAN with two-step masking for unsupervised time-series anomaly detection, but its review flags dataset, explanation, and generalizability gaps.The review also questions correctness and reproducibility because of unclear presentation.Source: B Sample Reviews · pp. 13–14
Sample review feedback
- A sample review scored 5 (marginally below acceptance), rated broader-impact discussion only partially addressed, and requested clearer research framing, dataset context, and scalability discussion.The review also asked for stronger literature connections and broader evaluation.Source: B Sample Reviews · p. 14
Influence functions
- The influence-functions review praises applications and scalability, while noting differentiability and convexity assumptions and that influence functions measure only local changes.It describes applications including model understanding, debugging, dataset-error detection, training-set attacks, and domain-mismatch improvement.Source: B Sample Reviews · p. 14
CHIMLE
- CHIMLE is credited with sample efficiency and quality, outperforming cIMLE and GAN- and diffusion-based methods across four tasks, but its outputs remain similar to training data.The review notes that TIM's modular design allows expansion, while greater model capacity could have environmental implications.Source: B Sample Reviews · pp. 15–16