Can large language models provide useful feedback on research papers? A large-scale empirical analysis

• Scientific feedback supports discovery, but publication growth and specialization make timely, high-quality reviews difficult and deepen inequalities in access.The introduction notes that marginalized researchers, especially those at non-elite institutions or in resource-limited regions, face disproportionate challenges accessing valuable feedback.Source: Introduction · p. 2
• The authors argue for scalable feedback mechanisms that could improve research and democratize access across the scientific community.Source: Introduction · p. 2
• Despite LLM applications in paper screening, error identification, and checklist verification, large-scale evidence on their role in scientific feedback is lacking.Source: Introduction · p. 2
• The study introduces a GPT-4 pipeline that turns paper PDFs into structured feedback on significance, acceptance and rejection reasons, and improvement suggestions.Source: Introduction · p. 2
• Retrospective analyses compare GPT-4 feedback with reviews of post-2022 Nature-family and ICLR papers, examining overlap in key points and topic distributions.The ICLR dataset includes complete reviews for accepted and rejected papers; the post-2022 cutoff avoids testing on data within GPT-4's training period.Source: Introduction · p. 2
• A prospective survey gathered feedback evaluations from 308 AI and computational-biology researchers at 110 US institutions.Researchers evaluated GPT-4 feedback on their own papers, including its helpfulness, reliability, limitations, and potential role in manuscript improvement.Source: Introduction · pp. 2–3

Generating Scientific Feedback using LLM

• The pipeline uses GPT-4 to generate feedback on full PDFs of scientific papers.Source: Generating Scientific Feedback using LLM · p. 3
• It first parses the entire paper from its PDF.Source: Generating Scientific Feedback using LLM · p. 3
• The system then constructs a paper-specific prompt for GPT-4.Source: Generating Scientific Feedback using LLM · p. 3
• The prompt concatenates designed instructions with the paper’s title, abstract, figure and table captions, and other main text.Source: Generating Scientific Feedback using LLM · p. 3
• GPT-4 receives the prompt and generates scientific feedback in a single pass.Source: Generating Scientific Feedback using LLM · p. 3
• Further pipeline details and validations are provided in the Supplementary Information.Source: Generating Scientific Feedback using LLM · p. 3

Retrospective Evaluation

• The retrospective evaluation covered 8,745 human comments on 3,096 accepted Nature-family papers and 6,505 comments on 1,709 ICLR papers; the matching pipeline’s extraction and matching F1 scores were 96.8% and 82.4%.The Nature dataset spanned 15 journals; ICLR included accepted and rejected papers.Source: Retrospective Evaluation · p. 3

LLM feedback significantly overlaps with human-generated feedback

• GPT-4’s comment overlap with an individual reviewer was 30.85% in Nature journals versus 28.58% between human reviewers, and 39.23% versus 35.25% in ICLR.The comparison controlled for the number of comments.Source: LLM feedback significantly overlaps… · pp. 3–4
• For ICLR, average GPT-4–human overlap rose from 30.63% for oral-accepted papers to 47.09% for rejected papers; human–human overlap rose from 23.54% to 43.80%.The oral and spotlight groups represent the top 5% and top 25% of accepted papers, respectively; overlap was also higher for spotlight papers than oral papers.Source: LLM feedback significantly overlaps… · p. 4

LLM could generate non-generic feedbacks.

• Shuffling GPT-4 feedback across papers reduced overlap from 30.85% to 0.43% in Nature journals and from 39.23% to 3.91% in ICLR, supporting paper-specific feedback.Source: LLM could generate non-generic feedbacks · p. 4

LLM is consistent with humans on major comments

• LLM comment identification rose with reviewer agreement: 11.39%, 20.67%, and 31.67% for 1, 2, and 3+ reviewers in Nature, and 15.39%, 26.21%, and 39.33% in ICLR.Comments raised by multiple reviewers were more likely to be echoed by the LLM.Source: LLM is consistent with humans on major… · p. 4
• In Nature reviews, first-quarter comments had 21.23% overlap with LLM comments versus 16.74% for the second quarter; ICLR showed the same tendency for earlier comments to overlap more.Source: LLM is consistent with humans on major… · p. 5

LLM feedback emphasizes certain aspects more than humans

• GPT-4 comments addressed research implications 7.27 times more often than humans, while humans were 10.69 times more likely to address novelty and 6.71 times more likely to request ablations.GPT-4 was 2.19 times more likely than humans to request experiments on more datasets, suggesting complementary emphases.Source: LLM feedback emphasizes certain… · p. 5

Prospective User Study and Survey

• More than 70% perceived at least partial alignment with their expectations, while 32.9% rated the feedback as more specific than some but less specific than many peer reviewers.Additionally, 35% considered alignment considerable or substantial; 17.3% rated specificity about equal to peer reviewers and 14% higher than many peer reviewers.Source: Prospective User Study and Survey · p. 5

Researchers find LLM feedback helpful

• A majority—over 50.3%—found the LLM feedback helpful and 7.1% very helpful; compared with human feedback, 41.9% rated it less helpful than many but more helpful than some.Additionally, 17.5% considered it inferior to human feedback, 20.1% about as helpful, and 20.4% more helpful.Source: Researchers find LLM feedback helpful · p. 5
• 50.5% were willing to reuse the system, and participants across educational and research-experience groups reported similar helpfulness and alignment.Participants also expressed optimism about improving human feedback and saw authors as more likely to benefit than reviewers or area chairs.Source: Researchers find LLM feedback helpful · p. 6

LLM could generate novel feedback not mentioned by humans.

• 65.3% thought LLM feedback offered perspectives overlooked or underemphasized by humans.Participants described it as identifying limitations and suggestions, including visualization and data privacy issues, that human reviewers had missed.Source: LLM could generate novel feedback not… · p. 6

Limitations of LLM feedback

• Participants identified insufficiently specific and actionable feedback as a key limitation, including a lack of domain-specific technical critique.They suggested prompting for concrete advice, such as specific missing work or experiments; some found the feedback more focused on testing regimes and presentation than ML details, while still useful for polishing papers.Source: Limitations of LLM feedback · p. 6

Discussion

• Researchers often found the feedback useful, and it sometimes raised points that human reviewers had not covered.The authors suggest it could support timely self-checking and may be especially helpful to researchers with limited access to quality feedback.Source: Discussion · p. 7
• Expert human feedback remains essential because the system can favor certain suggestions or feel generic, and cannot replace thoughtful domain-expert review.The authors note that some participants also described human reviews as generic.Source: Discussion · p. 7
• LLM feedback should help authors identify improvements before submission, not replace reviewers’ independent reading and assessment.The authors warn that generating reviews without thoroughly reading manuscripts would undermine rigorous evaluation.Source: Discussion · p. 7
• The study offers a preliminary prototype for exploring how responsibly deployed LLMs might change research, collaboration, and evaluation practices.The discussion situates this possibility within broader conversations about LLMs and generative AI’s potential effects on productivity, creativity, and scientific discovery.Source: Discussion · p. 7
• The results reflect one GPT-4, zero-shot pipeline and should be read as a lower bound, while the evaluation covers top English-language venues and a self-selected user sample.The authors propose exploring other models, fine-tuning, prompt engineering, and tools such as fact-checking or algorithm-analysis APIs; the current GPT-4 system also cannot interpret figures, graphs, or tables.Source: Discussion · pp. 7–8
• Future work could test whether LLM feedback detects and corrects different kinds of errors, assess false positives and negatives, and evaluate papers in other languages.The authors specifically propose testing typos, data-analysis mistakes, and mathematical-equation errors, as well as manuscripts by authors whose first language is not English.Source: Discussion · p. 8

Overview of the Nature Family Journals Dataset

• The Nature-family dataset included 3,096 accepted papers and 8,745 reviews published from January 1, 2022, to June 17, 2023.The papers came from 15 Nature-family journals.Source: Overview of the Nature Family Journals… · p. 8

Overview of the ICLR Dataset

• The ICLR dataset included 1,709 papers and 6,506 reviews from 2022 and 2023, sampled across oral, spotlight, poster, rejected, and withdrawn outcomes.The sampling was stratified by outcome and year.Source: Overview of the ICLR Dataset · p. 8

Generating Scientific Feedbacks using LLM

• The pipeline parsed paper PDFs with ScienceBeam and gave GPT-4 the first 6,500 extracted tokens to generate a single-pass outline covering four reviewer-feedback sections.The sections were significance and novelty, potential reasons for acceptance, potential reasons for rejection, and suggestions for improvement.Source: Generating Scientific Feedbacks using… · pp. 8–9

Retrospective Extraction and Matching of Comments from Scientific Feedback

• The matching pipeline extracted criticism points from LLM and human feedback, then used GPT-4 to identify semantic matches, retaining only matches rated 7 or higher.GPT-4 rated match similarity from 5 to 10; matches rated 5 or 6 were excluded from subsequent analyses.Source: Retrospective Extraction and Matching… · p. 9
• Extraction achieved F1 0.968 (precision 0.977, recall 0.960), while semantic matching achieved F1 0.824 (precision 0.777, recall 0.878).For semantic matching, annotators had 89.8% pairwise agreement and an F1 score of 88.7% against majority consensus.Source: Retrospective Extraction and Matching… · p. 9

Evaluating Specificity of LLM Feedback through Review Shuffling

• The shuffling test paired LLM feedback with a different paper’s human feedback from the same Nature journal and category or ICLR year to test whether overlap fell.The test evaluated the null hypothesis that generic LLM feedback would show little drop in overlap after shuffling.Source: Evaluating Specificity of LLM Feedback… · pp. 9–10

Overlap Metrics for Retrospective Evaluations and Control

• The retrospective analysis measured comment hit rate for GPT-4–human and human–human pairs, controlled human comments to GPT-4’s count, and found comparable overlap across alternative metrics.The analysis also evaluated the Szymkiewicz–Simpson overlap coefficient, Jaccard index, and Sørensen–Dice coefficient.Source: Overlap Metrics for Retrospective… · p. 10

Characterizing the comment aspects in human and LLM feedback

• The ICLR aspect analysis annotated LLM and human comments against an 11-aspect schema on 500 sampled papers, while prospective participants uploaded post-September 2021 PDFs and completed a survey.The 15–20-minute survey covered author backgrounds and evaluations of LLM reviews; participants were recruited through institute mailing lists and author outreach, compensated $20, and the study had Stanford IRB approval.Source: Characterizing the comment aspects in… · pp. 10–11

Additional Related Work

• Earlier work explored AI-assisted peer review, including small-scale studies of ChatGPT and GPT-4 and analyses of paper errors, quality, and gender disparities [60] [26] [25] [61].Other tools summarized papers, detected inaccurately reported p-values, rectified citation errors, and identified fairness disparities [56] [57] [58] [59].Source: Additional Related Work · p. 19
• The study combines 3,096 Nature-family papers, 1,709 ICLR papers, and 308 prospective-study responses from 110 US institutions in a systematic quantitative assessment.The assessment includes overlap analyses and human-evaluation metrics to examine the promise and limitations of LLM scientific feedback.Source: Additional Related Work · p. 19

F1 Score 0.968

• 2,634 true positives, 110 false negatives, and 63 false positives were recorded in human verification of the extractive summarization stage.The supplementary table also reports high accuracy for semantic matching after annotators reviewed 12,035 comment pairs.Source: Additional Related Work · p. 22, Table 3 · p. 22