Source-linked AI summary
Can large language models provide useful feedback on research papers? A large-scale empirical analysis
Weixin Liang, Yuhui Zhang, Hancheng Cao, Binglu Wang, Daisy Ding, Xinyu Yang, Kailas Vodrahalli, Siyu He, Daniel Smith, Yian Yin, Daniel McFarland, James Zou
TL;DR
Scientific feedback is difficult to obtain reliably at scale, motivating empirical study of LLM-generated feedback. This paper builds a GPT-4 pipeline for full-paper feedback and evaluates it through retrospective reviewer comparisons and a prospective user study. The authors report substantial overlap with human feedback and generally positive researcher perceptions, while identifying limits in specificity and expert-level critique.
Problem
Large-scale empirical evidence is lacking on whether and how LLMs can facilitate scientific feedback and augment academic practices.
Method
The study builds a GPT-4 pipeline for generating structured feedback from full paper PDFs and evaluates it retrospectively against human reviews and prospectively with researchers.
Results
GPT-4 feedback showed substantial overlap with human feedback, and a majority of participants regarded it as useful in manuscript improvement.
Takeaways & Limitations
LLM feedback may provide useful additional perspectives in manuscript improvement, including novel points not covered by human reviewers.
Takeaways & Limitations
The current system often produces generic or insufficiently specific feedback and cannot substitute for thoughtful domain-expert human feedback.
Abstract
from arXiv · showhide
Expert feedback lays the foundation of rigorous research. However, the rapid growth of scholarly production and intricate knowledge specialization challenge the conventional scientific feedback mechanisms. High-quality peer reviews are increasingly difficult to obtain. Researchers who are more junior or from under-resourced settings have especially hard times getting timely feedback. With the breakthrough of large language models (LLM) such as GPT-4, there is growing interest in using LLMs to generate scientific feedback on research manuscripts. However, the utility of LLM-generated feedback has not been systematically studied. To address this gap, we created an automated pipeline using GPT-4 to provide comments on the full PDFs of scientific papers. We evaluated the quality of GPT-4's feedback through two large-scale studies. We first quantitatively compared GPT-4's generated feedback with human peer reviewer feedback in 15 Nature family journals (3,096 papers in total) and the ICLR machine learning conference (1,709 papers). The overlap in the points raised by GPT-4 and by human reviewers (average overlap 30.85% for Nature journals, 39.23% for ICLR) is comparable to the overlap between two human reviewers (average overlap 28.58% for Nature journals, 35.25% for ICLR). The overlap between GPT-4 and human reviewers is larger for the weaker papers. We then conducted a prospective user study with 308 researchers from 110 US institutions in the field of AI and computational biology to understand how researchers perceive feedback generated by our GPT-4 system on their own papers. Overall, more than half (57.4%) of the users found GPT-4 generated feedback helpful/very helpful and 82.4% found it more beneficial than feedback from at least some human reviewers. While our findings show that LLM-generated feedback can help researchers, we also identify several limitations.
Introduction
Scientific feedback is increasingly difficult to obtain at scale, while the usefulness of LLM-generated feedback remains insufficiently established. This study introduces a GPT-4 pipeline and evaluates its feedback against human perspectives.
- Peer feedback supports scientific discovery and communication by connecting insights, coordinating research, and stimulating constructive debate.
- Growing publication volume and specialization make peer review laborious, resource-intensive, and constrained in scalability, expertise accessibility, and promptness.
- Large language models offer potential for scientific feedback, but large-scale empirical evidence on their utility and reliability remains limited.
- The study develops a GPT-4 pipeline that generates structured feedback on papers across significance, acceptance and rejection reasons, and improvement suggestions.
- The evaluation combines retrospective comparisons with human reviews and a prospective survey of researchers assessing feedback on their own papers.
Generating Scientific Feedback using LLM
The system processes complete scientific-paper PDFs and uses GPT-4 to generate structured feedback from paper-specific content in a single pass.
- The automated pipeline parses the full paper PDF and constructs a paper-specific prompt for GPT-4.
- The prompt combines designed instructions with the title, abstract, figure and table captions, and other main text.
- GPT-4 generates the scientific feedback in a single pass after receiving the constructed prompt.
Retrospective Evaluation
Retrospective analyses compare GPT-4 feedback with human reviews across Nature family journals and ICLR, measuring shared comments and their thematic patterns. GPT-4–human overlap is broadly comparable to human–human overlap and is higher for weaker papers, while the systems differ in emphasis.
- Datasets and evaluation: 3,096 accepted papers across 15 Nature family journals contributed 8,745 human-review comments, while 1,709 ICLR papers contributed 6,505 comments.
- Datasets and evaluation: 96.8% F1 was achieved for extractive comment extraction before semantic matching identified shared GPT-4 and human feedback points.
- Nature family journals: 30.85% of GPT-4 comments overlapped with an individual Nature reviewer, compared with 28.58% between two human reviewers.
- ICLR: 39.23% of GPT-4 comments overlapped with an individual ICLR reviewer, compared with 35.25% between two human reviewers.
- Paper quality: 47.09% overlap for rejected ICLR papers exceeded 30.63% for oral papers and 32.12% for spotlight papers.
- Comment characteristics: LLMs were more likely to echo comments raised by multiple reviewers: in Nature data, likelihood rose from 11.39% for one reviewer to 31.67% for three or more.
- Comment characteristics: Nature comments in the first review quarter had a 21.23% overlap likelihood, decreasing to 16.74% in the second quarter.
- Feedback aspects: LLMs discussed research implications 7.27 times more frequently than humans, but commented on novelty 10.69 times less often.
Prospective User Study and Survey
The prospective survey found generally positive perceptions of GPT-4 feedback, including alignment with human reviewers and suggestions of overlooked issues. Participants also identified a key limitation: feedback was often insufficiently specific and actionable, especially for domain-specific technical improvements.
- 308 researchers from 110 US institutions evaluated GPT-4 feedback on improving their work or understanding a subject.
- More than 70% of participants perceived at least partial alignment with the significant points and issues expected for their papers.
- 50.5% of participants expressed willingness to reuse the system, and perceptions were similar across demographic and research-experience groups.
- 65.3% of participants thought GPT-4 offered perspectives overlooked or underemphasized by humans, including visualization and data-privacy suggestions.
- Participants’ main criticism was that feedback lacked specific, actionable, and domain-specific recommendations, including in-depth critique of model architecture and design.
- Participants noted that GPT-generated reviews focused less on content and ML details but still offered actionable advice on paper layout, presentation, and testing regimes.
Discussion
The discussion presents GPT-4 feedback as a potentially useful resource that can complement scientific evaluation, particularly for researchers lacking timely access to expert feedback. It also stresses that current systems remain limited in specificity, scope, and appropriate use, and cannot replace domain-expert review.
- LLM feedback showed substantial overlap with human feedback and was generally viewed as useful for manuscript improvement, sometimes raising points humans missed.
- The system may be especially valuable for researchers lacking access to timely, high-quality feedback, including those from traditionally underprivileged regions.
- Expert human feedback remains central to rigorous scientific evaluation because current LLM feedback can be generic, biased toward certain aspects, and unable to substitute for specific domain expertise.
- LLM feedback should primarily help authors identify manuscript improvements before submission, while expert reviewers should independently read and assess manuscripts.
- The study frames its prototype as a preliminary investigation into how responsibly deployed generative AI might affect research, collaboration, and evaluation practices.
- The evidence reflects one GPT-4 model and prompt using zero-shot learning, so the results are interpreted as a lower bound on LLM feedback potential.
- The dataset focuses on rejected papers and top English-language venues, while the user study covers machine learning and computational biology and has limited population coverage and self-selection concerns.
- Future work could test whether LLM feedback detects and corrects typos, data-analysis mistakes, and mathematical-equation errors.
Methods
The study evaluates GPT-4-generated scientific feedback through retrospective comparison with human reviews and a prospective user study. Its pipelines generate structured feedback, extract and match comments, and assess robustness across datasets and overlap metrics.
- Retrospective evaluation: 1709 ICLR papers and 3,096 papers from 15 Nature family journals formed the retrospective evaluation datasets.The ICLR sample included papers and reviews from 2022 and 2023, grouped by decision outcomes.
- Feedback generation: GPT-4 generated structured feedback from paper PDFs using extracted manuscript content and reviewer-style instructions covering four feedback sections.The sections addressed significance and novelty, reasons for acceptance, reasons for rejection, and suggestions for improvement.
- Comment matching: The retrospective matching pipeline extracted key comments with GPT-4 and then used semantic matching to identify shared comments between LLM and human feedback.Human verification produced an F1 score of 0.968 for extraction, with precision 0.977 and recall 0.960.
- Overlap metrics: Hit rates compared comments in GPT-4-versus-human and human-versus-human feedback, while comment counts were controlled for direct comparison.Additional robustness checks used the Szymkiewicz–Simpson overlap coefficient, Jaccard index, and Sørensen–Dice coefficient.
- Robustness analysis: The overlap between GPT-4 and human feedback remained comparable to human-versus-human overlap across additional set-overlap metrics.Results were also largely similar with and without comment-count control across the ICLR and Nature family datasets.
- Prospective user study: A prospective online study delivered GPT-4 feedback by email after users uploaded papers and evaluated perceptions through a six-part survey.The survey covered author background, review contexts, impressions, detailed evaluations, comparisons with human reviews, and additional feedback.
Code Availability
The authors provide the implementation code for the scientific-feedback system and analyses through a public GitHub repository.
- Code Availability: The study’s code is available at the authors’ GitHub repository.Repository: https://github.com/Weixin-Liang/LLM-scientific-feedback.
Supplementary Information
The supplementary materials document related work, datasets, validation results, token lengths, example extracted comments, and robustness analyses for the feedback evaluation.
- Additional Related Work: The study distinguishes its work from prior exploratory efforts through large-scale empirical analysis and a prospective user study.The datasets include 3,096 Nature-family papers, 1,709 ICLR papers, and 308 responses from 110 US institutions.
- Pipeline validation: Supplementary Table 3 reports human verification of comment extraction and semantic matching using sampled feedback and comment pairs.The validation samples included 639 scientific feedbacks for extraction and 12,035 extracted-comment pairs for matching.
- Comment examples: Supplementary Tables 5–7 provide examples of comments extracted from LLM and human feedback on ICLR papers by human coding.These examples support inspection of the comment representation used in the retrospective analysis.
- Supplementary figures: Supplementary analyses show that GPT-4 feedback overlaps substantially with human feedback and that conclusions are robust across alternative set-overlap metrics.Additional figures report overlap, hit-rate controls, correlations, and perceived helpfulness across participant groups.