Source-linked AI summary
Generalization Bias in Large Language Model Summarization of Scientific Research
Uwe Peters, Benjamin Chin-Yee
TL;DR
LLMs may omit qualifiers and limitations when summarizing scientific research, potentially broadening conclusions beyond the original evidence. This study systematically evaluated prominent LLM summaries against their source texts and found persistent overgeneralization, including stronger tendencies among newer models.
Problem
Whether LLMs accurately preserve the scope of scientific research conclusions remains underexamined, despite concerns that omitted qualifiers and limitations can produce unwarranted overgeneralizations.
Method
The study used logistic regression and comparisons between original scientific texts and nearly 5000 summaries generated by prominent LLMs to assess generalized conclusions and algorithmic overgeneralization.
Results
Most tested LLMs produced broader scientific generalizations than the original texts, with several models reliably making broader claims and newer models generally showing greater inaccuracies.
Takeaways & Limitations
The findings indicate a persistent generalization bias in many LLMs that can extend scientific claims beyond the source material, underscoring the need for stronger safeguards.
Takeaways & Limitations
Most analyses focused on medical abstracts and full-length medical articles, and the human-versus-LLM full-article comparison included only three LLM systems and one human-summary source.
Abstract
from arXiv · showhide
Artificial intelligence chatbots driven by large language models (LLMs) have the potential to increase public science literacy and support scientific research, as they can quickly summarize complex scientific information in accessible terms. However, when summarizing scientific texts, LLMs may omit details that limit the scope of research conclusions, leading to generalizations of results broader than warranted by the original study. We tested 10 prominent LLMs, including ChatGPT-4o, ChatGPT-4.5, DeepSeek, LLaMA 3.3 70B, and Claude 3.7 Sonnet, comparing 4900 LLM-generated summaries to their original scientific texts. Even when explicitly prompted for accuracy, most LLMs produced broader generalizations of scientific results than those in the original texts, with DeepSeek, ChatGPT-4o, and LLaMA 3.3 70B overgeneralizing in 26 to 73% of cases. In a direct comparison of LLM-generated and human-authored science summaries, LLM summaries were nearly five times more likely to contain broad generalizations (OR = 4.85, 95% CI [3.06, 7.70]). Notably, newer models tended to perform worse in generalization accuracy than earlier ones. Our results indicate a strong bias in many widely used LLMs towards overgeneralizing scientific conclusions, posing a significant risk of large-scale misinterpretations of research findings. We highlight potential mitigation strategies, including lowering LLM temperature settings and benchmarking LLMs for generalization accuracy.
1. Introduction
LLMs can make scientific findings more accessible, but may broaden conclusions by omitting qualifiers and changing how results are generalized. This study examines these risks across prominent models and defines measurable forms of overgeneralization.
- Motivation: LLM chatbots can summarize complex scientific information accessibly, supporting science literacy, policy, behavior, and research.
- Motivation: Omitting uncertainties, limitations, and nuances can broaden scientific claims beyond what original findings warrant.
- Motivation: Broader chatbot claims may contribute to widespread misunderstandings and, in clinical settings, unsafe or inappropriate treatments.
- Research gap: The study addresses whether LLMs faithfully capture scientific generalizations, a question left unexamined despite documented human overgeneralization.
- Study design: The researchers tested 10 prominent LLMs on abstracts and selected full-length articles from leading science and medical journals.
- Generalization types: Generic statements can obscure variation because their scope is semantically underdetermined, potentially converting quantified findings into broader claims.
- Generalization types: Present-tense wording extends claims beyond a particular sample, while action-guiding recommendations may exceed the evidence supporting the descriptive result.
- Operationalization: The analysis classifies conclusions as generalized when they shift from quantified, past-tense, or descriptive claims to generic, present-tense, or action-guiding claims.
2. Results
Across models, LLM summaries more often contained generalized conclusions than source texts, with especially high rates among several newer models. Lower temperature reduced generalized outputs, whereas an accuracy-focused prompt unexpectedly increased them.
- Overall tendency: LLM summaries were twice as likely as original abstracts to contain generalized conclusions.The combined regression model was significant: F7, 4492 = 32.34, p < 0.001.
- Model differences: Six of 10 models were significantly more likely than original texts to produce generalized conclusions.Older models showed about 2.6-fold increases, versus 9-fold for ChatGPT-4o and 39-fold for LLaMA 3.3 70B.
- Temperature: At temperature 0, generalized conclusions were 76% less likely than at temperature 0.7.
- Model differences: Overall algorithmic overgeneralization reached 45−60% for ChatGPT-4o, 69−73% for LLaMA 3.3 70B, and 26−67% for DeepSeek.Claude models had the lowest range, from −1 to 20%.
- Model differences: Specific overgeneralization reached 61% for ChatGPT-4o and 73% for LLaMA 3.3 70B, while Claude consistently had the lowest proportion.
- Prompt effects: Accuracy-focused prompting increased generalized conclusions relative to a simple prompt, with OR = 1.90, 95% CI [1.11, 3.26], p = 0.02.The systematic prompt did not significantly change the likelihood.
- Human comparison: LLM article summaries had OR = 6.72, 95% CI [4.22, 10.71], p < 0.001, for generalized conclusions versus original articles.
3. Discussion
The study identifies frequent algorithmic overgeneralization in LLM scientific summaries, including broader claims than original texts and stronger tendencies in several newer models. Accuracy-focused prompting unexpectedly increased overgeneralization, while the pattern was robust on retesting and exceeded human-summary rates for some models.
- Prompt effects: Requesting faithful summaries doubled the likelihood of generalized conclusions and increased overall overgeneralizations by up to 15% in some models.The authors suggest the wording ‘do not introduce any inaccuracies’ may contribute to this backfire effect.
- Robustness and human comparison: Overgeneralization was robust on retesting, and GPT-4 Turbo, ChatGPT-4o, and DeepSeek were more prone to it than human NEJM JW authors.The comparison used NEJM JW summaries of the same articles; those authors were domain experts writing for an expert audience.
4. Recommendations
The authors propose mitigation strategies focused on generation settings, prompt design, model selection, prompt transformation, and systematic benchmarking. These recommendations aim to reduce or detect cases where summaries broaden scientific claims beyond their original scope.
- Generation settings: Use temperature 0 for high-stakes scientific summarization when the interface or platform permits temperature control.The study found that lower temperatures reduced algorithmic overgeneralization.
- Prompt design: Avoid direct accuracy prompts that ask LLMs not to introduce inaccuracies, because such phrasing increased algorithmic overgeneralizations.The recommendation follows the observed backfire effect of accuracy-focused prompting.
- Model selection: Consider Claude or older LLMs for scientific summarization because their outputs remained closer to the original texts in generalization scope.ChatGPT-4.5 was an exception among newer models because it was still a Research Preview.
- Prompt transformation: Use shadow prompting to transform user prompts into past-tense scientific summaries, which could prevent inaccurate generalization.The proposed transformation would be implemented by developers before response generation.
- Benchmarking: Benchmark LLMs with a three-step framework that compares coded generalization features in original texts and generated summaries.The framework prompts summaries, classifies generic, present-tense, and action-guiding generalizations, and detects broadened claims.
5. Strengths and limitations
The study combines a large, multi-model evaluation with diachronic comparisons, but its evidence is constrained by limited prompt coverage, model selection, medical emphasis, and a primary focus on overgeneralization.
- Strengths: 4900 LLM-generated summaries and 10 influential models, tested across temperatures and prompts, provide broad coverage and diachronic insight into generalization performance.The study included earlier and newer models, including DeepSeek, ChatGPT-4o, ChatGPT-4.5, and Claude 3.7 Sonnet.
- Limitations: Only three prompts were tested, so broader prompt engineering is needed to identify effective mitigation strategies.The prompts were chosen based on intuitiveness or existing literature.
- Limitations: The model sample was incomplete, and the human comparison used only three LLMs against NEJM JW summaries, which may not represent all human science summarization.Other LLMs may be more accurate, and other models could be used for science summarization.
- Limitations: Most analyses focused on medical abstracts and full-length medical articles, so generalization tendencies may vary across scientific subject areas.The authors call for future research across different scientific text domains.
- Limitations: The study focused on overly broad generalizations, although errors can also occur through undergeneralization.Overgeneralizations were reported as far more frequent, motivating the primary focus on them.
6. Conclusion
The study systematically evaluates whether prominent LLMs preserve the scope of scientific claims or exaggerate it. Across nearly 5000 summaries, most models generalized findings more broadly than the source texts, with newer models showing greater inaccuracies.
- Conclusion: Most models produced broader scientific-result generalizations than the original texts, even when explicitly prompted for accuracy and across multiple tests.The study describes this pattern as a persistent generalization bias in many LLMs.
- Conclusion: Newer models exhibited significantly greater inaccuracies in generalization than earlier versions.The comparison provides a diachronic assessment of LLM generalization behaviour.
- Conclusion: The findings underscore the need for stronger safeguards in AI-driven science summarization to reduce widespread misunderstandings of research.The paper defines generalization bias as extrapolating results beyond claims in the summarized material.
7. Methods
The preregistered experimental study tested 10 LLMs on scientific and medical texts, repeatedly retrieved summaries under varied conditions, and coded whether conclusions were restricted or generalized.
- Design: The study tested 10 LLMs through APIs or user interfaces, combining between-subject and within-subject designs with repeated testing of some models.Separate chats, new accounts, or disabled memory were used for UI collections to mitigate personalization.
- Materials: 200 abstracts were sampled from leading medical and multidisciplinary science journals, with an additional 100 full-length prospective clinical articles.The article sample focused on original clinical studies relevant to intervention efficacy.
- Materials: Article summaries from GPT-4 Turbo, ChatGPT-4o, and DeepSeek were retrieved with a standardized title-producing prompt for comparison with NEJM JW summaries.The prompt was designed to improve comparability with the human-authored summaries.
- Coding: Two experts coded summaries as containing restricted or generalized conclusions using preregistered criteria, with blinded third-researcher checks yielding k = 0.79 to 0.95 agreement.Disagreements were resolved through discussion.
- Analysis: Generalized-conclusion probabilities were analyzed with binomial-logit generalized linear mixed models, using fixed experimental factors and random effects for repeated texts.Separate models and Bonferroni corrections addressed different text types and multicollinearity concerns.
- Analysis: Model assumptions were assessed and met, including independence, logit linearity, acceptable multicollinearity, and random-effect significance.Original-text identifiers were included as random effects to support independence.
Data availability
The study reports that its data and LLM responses are available on an OSF platform.
- Data availability: All data and LLM responses are available on an OSF platform.
- Authorship: The study identifies U.P. as the main author and describes contributions spanning design, data collection, analysis, interpretation, drafting, and revision.