Source-linked AI summary
Blind to the Human Touch: Overlap Bias in LLM-Based Summary Evaluation
Jiangnan Fang, Cheng-Tse Liu, Hanieh Deilamsalehy, Nesreen K. Ahmed, Puneet Mathur, Nedim Lipka, Franck Dernoncourt, Ryan A. Rossi
TL;DR
The paper examines how LLM-judge preferences relate to n-gram overlap with human summaries and whether this pattern is consistent across models and position biases. Across nine models, generated summaries are preferred more strongly as overlap decreases, while position-bias differences persist; the study is limited by its metrics, references, length range, and lack of adversarial examples.
Problem
The paper asks how LLM judgments correlate with n-gram-based similarity metrics and whether this relationship is consistent across models of different sizes.
Method
The study evaluates nine decoder-only LLMs across WikiSum and CNN_DailyMail, using BLEU and ROUGE overlap scores against human summaries.
Results
LLM judges generally prefer generated summaries more strongly when their overlap with human summaries is lower, and this pattern persists across position-bias types and model sizes.
Takeaways & Limitations
LLM-as-a-judge evaluation of summaries should use techniques beyond simple comparison because models struggle even when summaries have limited overlap.
Takeaways & Limitations
The study uses short-phrase overlap metrics, one reference summary, a restricted 95–105-word subset, and no adversarial examples.
Abstract
from arXiv · showhide
Large language model (LLM) judges have often been used alongside traditional, algorithm-based metrics for tasks like summarization because they better capture semantic information, are better at reasoning, and are more robust to paraphrasing. However, LLM judges show biases for length and order among others, and are vulnerable to various adversarial input prompts. While recent studies have looked into these biases, few have analyzed them at a more granular level in relation to a well-defined overlap metric. In this work we provide an LLM judge bias analysis as a function of overlap with human-written responses in the domain of summarization. We test 9 recent LLMs with parameter counts ranging from 1 billion to 12 billion, including variants of Gemma 3 and LLaMA 3. We find that LLM judges increasingly prefer summaries generated by other LLMs over those written by humans as the similarities (as measured by ROUGE and BLEU) between the judged summaries decrease, and this pattern extends to all but one model tested, and exists regardless of the models' own position biases. Additionally, we find that models struggle to judge even summaries with limited overlaps, suggesting that LLM-as-a-judge in the summary domain should rely on techniques beyond a simple comparison.
1. Related Works
Prior work finds that LLM judges exhibit content-, authorship-, length-, and position-related biases, motivating systematic study of these tendencies in evaluation.
- Evaluation frameworks: Recent LLM-judge research develops evaluation datasets, judging frameworks, and prompting methods for open-ended and summarization tasks.Examples include scalable judge fine-tuning, personalized long-text evaluation, rubric-based response weighting, and chain-of-thought prompting.
- Content-level biases: LLM judges can overlook factual errors and exhibit gender, authority, beauty, and phrase-level content biases during evaluation.These findings include difficulties evaluating factual information in summarization.
- Authorship and length biases: LLM judges have also been reported to favor longer responses, outputs from other LLMs, and texts produced by the same model.These biases occur across comparisons involving human outputs and outputs from different AI systems.
2. Methodology
The study generates and evaluates summaries from multiple LLMs, measures overlap with human summaries, controls length and order effects, and expands the similarity range with rephrased inputs.
- Similarity measurement: Similarity to human summaries is scored using the average of BLEU-1, BLEU-4, ROUGE-1, and ROUGE-2.These overlap metrics quantify the generated summary’s similarity to its human reference.
- Summary generation and evaluation: LLMs first generate summaries, after which evaluator models judge human-generated and machine-generated summary pairs presented with the original texts.The evaluator prompt assigns the two summaries to alternative positions and uses no conversation history.
- Bias controls: Reference summaries are filtered to 95–105 space-delimited words to reduce length effects, while evaluations in both presentation orders control ordering bias.Inconsistent choices across orders are marked as ties and further classified by first- or last-position preference.
- Similarity-range extension: Because initial generated summaries had average similarity scores below 0.55, additional summaries are created by rephrasing and reorganizing ground-truth summaries while retaining longer expressions and segments.These additional summaries broaden the ROUGE and BLEU score range and are evaluated without revealing their rephrased origin.
3. Results & Discussion
Across most tested models, judges increasingly favor generated summaries as their n-gram overlap with human summaries decreases, while this preference persists across differing position biases and model sizes.
- RQ1: Similarity and judgment: For all models, human summaries are rarely chosen as better regardless of their similarity to human summaries.Except for model variants and the smallest Gemma-3-1B-it, lower overlap is associated with stronger preference for generated summaries.
- RQ1: Similarity and judgment: LLM judges show stronger AI-AI bias when generated summaries contain fewer n-gram overlaps with human-written summaries.The pattern appears even when the generated summaries come from smaller models.
- RQ1: Similarity and judgment: A significant non-overlap is required for generated-summary bias to appear; Mistral’s preference frequency drops below 25% for mean scores above 0.5.The reported threshold illustrates that the bias diminishes before similarity scores approach 1.
- RQ2: Position bias: Position bias is stronger when generated summaries more closely resemble human-written summaries.This is reflected by a larger proportion of tied choices at higher similarity values.
- RQ2: Position bias: Models with more parameters tend to prefer last-presented summaries, whereas models with fewer parameters tend to prefer first-presented summaries.Despite these different position-bias patterns, generated-summary preference persists across tested model sizes.
- Architecture and scale: Generated-summary preferences are similar among models sharing an architecture once they exceed a certain parameter count.The study gives the larger LLaMA and Gemma models as examples.
4. Conclusion
The study links human–machine summary similarity to LLM-judge preferences in summarization and finds a persistent preference for generated summaries at lower overlap, independent of position-bias type.
- Main findings: LLM judges prefer generated summaries over human texts, with stronger preference when overlap-metric similarity is lower.This preference extends to summaries generated by models with as few as 1 billion parameters.
- Main findings: Generated-summary bias persists regardless of whether a model prefers the first or last summary when both orders are tested.The conclusion identifies this pattern across varied model training techniques and training data.
- Implications: The observed bias may indicate a stylistic marker in LLM output that could be useful for LLM detection but unproductive for automatic evaluation.The authors also identify these bias patterns as potential targets for improving LLM training and judging frameworks.
5. Limitations
The study’s limitations primarily concern how similarity and evaluation scope were operationalized, along with restricted summary-length and robustness settings.
- The study uses short-phrase overlap metrics as a crude approximation of similarity to human-written texts.The authors suggest evaluating additional metrics as independent variables in future work.
- Only one reference text was used despite generating summaries with 9 models, limiting robustness of the observed bias patterns.The authors propose using multiple diverse human-written summaries.
- Filtering human summaries to 95–105 space-delimited words restricts findings to a subset of the datasets.The authors also leave adversarial examples unexamined, although such examples may disrupt the observed bias patterns.