Source-linked AI summary
The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinism
Yifan Song, Guoyin Wang, Sujian Li, Bill Yuchen Lin
TL;DR
LLM evaluations commonly rely on one output despite generation non-determinism, leaving performance variability insufficiently characterized. The paper evaluates greedy decoding and sampling across benchmarks, models, and settings, finding that greedy decoding generally performs better while alignment can reduce sampling variance and best-of-N sampling can unlock strong performance from smaller models.
Problem
Most LLM evaluations use a single output per example because inference and evaluation are costly, limiting assessment of generation non-determinism and variability.
Method
The paper evaluates greedy decoding and sampling across seven benchmarks, multiple LLM sizes and families, alignment methods, and generation settings.
Results
Greedy decoding generally outperforms sampling on most evaluated benchmarks, while alignment can reduce sampling variance and best-of-N sampling enables 7B-level models to potentially outperform GPT-4-Turbo.
Takeaways & Limitations
Non-determinism should be considered in LLM evaluation, and best-of-N results can help assess reward models.
Takeaways & Limitations
The paper leaves the mechanisms behind models’ inverse task-specific behaviors as questions for future research.
Abstract
from arXiv · showhide
Current evaluations of large language models (LLMs) often overlook non-determinism, typically focusing on a single output per example. This limits our understanding of LLM performance variability in real-world applications. Our study addresses this issue by exploring key questions about the performance differences between greedy decoding and sampling, identifying benchmarks' consistency regarding non-determinism, and examining unique model behaviors. Through extensive experiments, we observe that greedy decoding generally outperforms sampling methods for most evaluated tasks. We also observe consistent performance across different LLM sizes and alignment methods, noting that alignment can reduce sampling variance. Moreover, our best-of-N sampling approach demonstrates that smaller LLMs can match or surpass larger models such as GPT-4-Turbo, highlighting the untapped potential of smaller LLMs. This research shows the importance of considering non-determinism in LLM evaluations and provides insights for future LLM development and evaluation.
1 Introduction
LLM evaluations often use one output per example despite decoding-induced non-determinism, limiting understanding of performance variability. This study compares greedy decoding and sampling across benchmarks, model properties, and generation settings.
- Single-output evaluations overlook that the same LLM can produce significantly different outputs under greedy decoding and nucleus sampling.This practice is driven mainly by the computational cost of inference and evaluation.
- Omitting standard deviations makes LLM performance variability and real-world generation dynamics difficult to measure.
- The study asks when greedy decoding or sampling performs better, which benchmarks are most consistent, and whether models exhibit unique non-deterministic patterns.It also examines scaling, alignment, temperature, repetition, and best-of-N sampling.
- Across most evaluated benchmarks, greedy generation differs notably from the average score of multiple sampling runs, sometimes changing performance rankings.
- Greedy decoding generally outperforms sampling, except on AlpacaEval, while constrained-output benchmarks show more consistent performance across configurations.Math reasoning and code generation are especially affected by sampling variance.
2 Experimental Setup
The experiments compare greedy decoding with temperature- and top-p-controlled sampling across diverse instruction-following, knowledge, math, and coding benchmarks. They evaluate multiple model families and sizes, use benchmark-specific metrics and scripts, and report sampling variability across repeated completions.
- The benchmark suite covers instruction following, knowledge, math reasoning, and coding through AlpacaEval 2, Arena-Hard, WildBench v2, MixEval, MMLU-Redux, GSM8K, and HumanEval.
- AlpacaEval 2, Arena-Hard, and WildBench v2 use length-controlled win rate, baseline win rate, and taskwise WB-Score metrics, respectively.The selected datasets contain 805, 500, and 1024 questions or tasks, respectively.
- MMLU-Redux contains 3000 manually re-annotated questions across 30 subjects, addressing the original benchmark’s size and ground-truth-error concerns.
- The study tests open-weight Llama-3-Instruct, Yi-1.5-Chat, Qwen2-Instruct, and Mistral models alongside GPT-4-Turbo, including multiple sizes and alignment methods.
- The main comparison uses greedy decoding versus sampling with temperature 1.0 and top-p 1.0, evaluated through official or specified benchmark frameworks.Sampling uses 16 completions for four benchmarks, 32 for MMLU-Redux, and 128 for GSM8K and HumanEval.
- Table 2 reports sampling averages, standard deviations, and best-to-worst-run gaps, while Table 3 reports WildBench WB-Score and task-category scores.Table 3 uses 16 sampled generations per model.
3 Experimental Results
Across benchmarks and models, decoding choice creates meaningful performance differences and can even reverse model rankings. Greedy decoding is usually strongest, while sampling is favored on AlpacaEval; benchmark stability tracks output-space constraints, and some models show distinctive behavior.
- Q1. Performance gap: Different decoding configurations can alter model rankings: Qwen2-7B is slightly better than Llama-3-8B with greedy decoding, whereas Llama-3-8B may outperform Qwen2-7B with sampling.
- Q2. Decoding preference: Greedy decoding outperforms sampling for most evaluated tasks and models, with AlpacaEval as a notable exception where sampling performs better.
- Q3. Benchmark consistency: MixEval and MMLU are most stable because their ground-truth-based and multiple-choice formats constrain the answer space.
- Q3. Benchmark consistency: The best-to-worst sampling gap can exceed 10.0 points on the less stable GSM8K and HumanEval benchmarks.
- Q4. Model characteristics: GPT-4-Turbo has smaller greedy-versus-sampling gaps and improved sampling quality, while Mistral-7B-Instruct-v0.2 and Llama-3-8B-Instruct show task-specific inverse patterns.
4 How Various Factors Influence Non-Determinism?
The paper examines how scaling, alignment, temperature, repetition penalties, and generation length affect non-deterministic performance. Results show that model size does not reliably reduce sampling variance, alignment can reduce variance, and decoding settings can substantially change outcomes.
- Scaling: Model sizes from 0.5B to 34B show consistent section-level findings, but parameter scaling does not produce lower sampling variance.Qwen2-7B-Instruct shows higher variance on AlpacaEval and HumanEval than some other models.
- Alignment: Alignment methods can reduce sampling standard deviation across tasks including AlpacaEval, MMLU, GSM8K, and HumanEval.KTO and SimPO nevertheless reduce MMLU performance, and SimPO has limited effectiveness on MixEval.
- Temperature: Higher temperature slightly improves AlpacaEval, has little statistically significant effect on MMLU, and strongly affects reasoning and code generation at 1.5.The passage contrasts open-ended instruction following with multiple-choice question answering.
- Repetition penalty: Maintaining the default repetition penalty of 1.0 generally yields the best performance across evaluated settings.AlpacaEval improves marginally at 1.2, while repetition penalty has minimal impact on MixEval and MMLU.
- Surface patterns: Greedy decoding is usually marginally shorter than sampling, except for Yi models on AlpacaEval and GSM8K, where response lengths are comparable.The comparison concerns generation lengths across evaluated benchmarks.
- Case study: 83.5 vs. 72.0 on GSM8K: Qwen2-7B-Instruct performs substantially better with greedy decoding than sampling.Across 128 sampled responses for the same question, the error rate reaches 89%.
5 What is the Full Potential of Non-Determinism?
The paper evaluates whether sampling can reveal and exploit capabilities that are not reliably expressed by a single model output. Using Best-of-N selection with reward models and an oracle upper bound, it finds that smaller models can surpass GPT-4-Turbo on several benchmarks under oracle selection.
- Method: Best-of-N selects the highest-ranked response from N sampled outputs using off-the-shelf reward models, with oracle selection providing an upper bound.The approach directly addresses evaluations based on a single output per instance.
- Results: Best-of-N sampling produces significant performance enhancement for multiple sampled responses.The reported results concern Llama-3-8B-Instruct under reward-model and oracle selection.
- Results: Under oracle selection, Llama-3-8B-Instruct outperforms GPT-4-Turbo on MMLU, GSM8K, and HumanEval.The finding indicates that smaller models can contain strong capabilities that are not always decoded by standard generation.
- Future directions: The paper identifies probability calibration and selecting among multiple completions as directions for improving smaller LLMs.It also points to preference optimization, self-consistency, and advanced prompting as related avenues.
6 Related Work
Related work spans broad capability benchmarks, open-ended instruction-following evaluations, and focused math and code-generation tests. The paper positions its contribution as studying how decoding configurations and non-deterministic generation affect these evaluations.
- General benchmarks: MMLU, HellaSwag, and ARC evaluate knowledge understanding and complex reasoning across broad benchmark settings.These benchmarks expand evaluation beyond a single capability.
- Instruction following: AlpacaEval, MT-Bench, ArenaHard, and WildBench use frontier models as judges to evaluate open-ended instruction following.These evaluations target responses that are not limited to fixed answer choices.
- Math and code: GSM8K and MATH assess mathematical reasoning, while HumanEval and MBPP assess code generation.The paper evaluates non-determinism in settings represented by these benchmark families.
- Evaluation gap: Because inference and evaluation are costly, most LLM evaluations use a single output per example.This paper studies the impact of multiple generation configurations, particularly non-deterministic generation.
- Decoding strategies: Greedy decoding selects the highest-probability next token, whereas Top-k and Top-p sampling randomly select tokens from probability distributions.Temperature balances response quality and diversity, while length and repetition penalties also affect generation.
7 Conclusion & Future directions
The conclusion frames non-determinism as an overlooked dimension of LLM generation and summarizes evaluations across benchmarks, scaling, and alignment. It also highlights Best-of-N results as a useful reference for future reward-model research.
- Conclusion: The paper investigates overlooked questions about non-deterministic LLM generations across several evaluation dimensions.The stated goal is to improve understanding of generation methods and widely used benchmarks.
- Conclusion: Evaluations across seven commonly used benchmarks address research questions about generation behavior, scaling, and alignment.The paper presents these analyses as evidence for understanding non-deterministic generation.
- Future directions: Best-of-N results can serve as a benchmark for assessing reward models.The paper identifies this as one example of how its evaluation results may support future research.