PaperlayerView exact revision on arXiv

Source-linked AI summary

Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligence

Timothy R. McIntosh, Teo Susnjak, Nalin Arachchilage, Tong Liu, Paul Watters, Malka N. Halgamuge

arXiv:2402.09880v2cs.AIcs.CLcs.CYcs.HC

TL;DR

The paper addresses whether current LLM benchmarks adequately measure increasingly complex capabilities and risks. It reviews 23 state-of-the-art benchmarks with a unified people-process-technology framework focused on functionality and integrity, finding widespread limitations in reasoning, inclusivity, adaptability, implementation, and resistance to bias or gaming. It recommends moving beyond static task scores toward behavioral profiling and regular audits.

Abstract

from arXiv · show

The rapid rise in popularity of Large Language Models (LLMs) with emerging capabilities has spurred public curiosity to evaluate and compare different LLMs, leading many researchers to propose their own LLM benchmarks. Noticing preliminary inadequacies in those benchmarks, we embarked on a study to critically assess 23 state-of-the-art LLM benchmarks, using our novel unified evaluation framework through the lenses of people, process, and technology, under the pillars of benchmark functionality and integrity. Our research uncovered significant limitations, including biases, difficulties in measuring genuine reasoning, adaptability, implementation inconsistencies, prompt engineering complexity, evaluator diversity, and the overlooking of cultural and ideological norms in one comprehensive assessment. Our discussions emphasized the urgent need for standardized methodologies, regulatory certainties, and ethical guidelines in light of Artificial Intelligence (AI) advancements, including advocating for an evolution from static benchmarks to dynamic behavioral profiling to accurately capture LLMs' complex behaviors and potential risks. Our study highlighted the necessity for a paradigm shift in LLM evaluation methodologies, underlining the importance of collaborative efforts for the development of universally accepted benchmarks and the enhancement of AI systems' integration into society.

I. INTRODUCTION

Existing LLM benchmarks often emphasize standardized task performance while underrepresenting real-world applicability, safety, cultural diversity, and resistance to manipulation. The paper motivates a unified evaluation approach that addresses these gaps across people, process, and technology.

I. INTRODUCTION

  • Current LLM benchmarks often prioritize fixed answers and standard metrics, overlooking real-world applicability, human values, cultural variation, and safety.These limitations can leave models appearing capable in controlled settings while remaining vulnerable to bias, unsafe decisions, or manipulation.Source: §I · pp. 1–2, B. Benchmarking in Generative AI and… · p. 3
  • The study proposes evaluating benchmark functionality and integrity together, where functionality measures intended capabilities and integrity guards against gaming, overfitting, and misleading scores.Its unified framework is designed to support more effective and secure evaluation methods for real-world deployment.Source: §I · p. 2
  • The paper examines how standardized benchmarking can be extended to capture complex, evolving LLM capabilities and broader societal risks rather than only controlled task performance.Its stated contributions include a unified framework, a systematic critique of current benchmarks, and behavioral profiling with regular audits.Source: §I · p. 2

A. Traditional Benchmarking in Computer Science

  • Traditional computer-science benchmarks seek repeatable, objective comparisons, but benchmark manipulation can produce high scores without proportional real-world performance.Reliability also depends on reproducibility, generalizability, consistency, and objective measurement.Source: A. Traditional Benchmarking in… · pp. 2–3

B. Benchmarking in Generative AI and LLMs

  • Generative-AI benchmarks add challenges because open-ended outputs permit multiple valid answers, models are opaque, and evaluation often combines automatic metrics with subjective human judgment.These conditions complicate assessment of reasoning, coherence, relevance, ethics, bias, and accountability.Source: B. Benchmarking in Generative AI and… · p. 3
  • LLM benchmarking must account for evolving language and societal norms, data contamination, benchmark overfitting, linguistic diversity, cultural nuance, and ethical behavior.The paper therefore frames robust evaluation as a multidisciplinary problem involving linguistics, ethics, and social sciences.Source: B. Benchmarking in Generative AI and… · p. 3

III. UNIFIED EVALUATION FRAMEWORK FOR LLM BENCHMARKS

The paper develops a cybersecurity-informed framework that evaluates LLM benchmarks through people, process, and technology, while distinguishing functionality from integrity. It applies this framework to a structured review of relevant benchmarks and their evaluation practices.

III. UNIFIED EVALUATION FRAMEWORK FOR LLM BENCHMARKS

  • The proposed framework adapts cybersecurity risk-assessment principles to evaluate technical performance, applicability, robustness, and integrity in LLM benchmarks.Its purpose is broader than measuring task scores alone.Source: §III · p. 3, A. Applying the People, Process,… · p. 3

A. Applying the People, Process, Technology (PPT) Framework

  • The PPT lens covers evaluator and user diversity, standardized and adaptable procedures, and scalable technical infrastructure as complementary determinants of benchmark quality.These dimensions correspond to people, process, and technology respectively.Source: A. Applying the People, Process,… · pp. 3–4

B. Functionality and Integrity in LLM Benchmarks

  • Functionality asks whether a benchmark measures intended capabilities and real-world skills rather than superficial pattern matching, while integrity asks whether models can exploit metrics, memorization, or test-set characteristics.Together they target authentic assessment rather than inflated performance.Source: B. Functionality and Integrity in LLM… · p. 4

C. Data Collection

  • The study used a structured literature review of LLM benchmarks with multimodal capabilities, covering peer-reviewed work, technical preprints, and widely cited platforms through October 2023.Inclusion emphasized broad generative-AI tasks and substantial academic or technical adoption, while exclusions removed overly niche or poorly documented benchmarks.Source: C. Data Collection · p. 4

D. Data Analysis

The analysis systematically codes benchmark inadequacies in functionality and integrity, then surveys 23 benchmarks spanning general-purpose and specialized tasks. The resulting overview covers diverse evaluation methods, domains, languages, and model capabilities.

D. Data Analysis

  • The analysis identifies, categorizes, and assesses benchmark inadequacies in functionality and integrity, including response variability, overfitting, and limited linguistic diversity.Prevalence counts how many of the 23 benchmarks exhibit each inadequacy, while creator responses are classified as unacknowledged, unresolved, or addressed.Source: D. Data Analysis · pp. 4–5
  • Figure 1 presents the evaluation flowchart guiding the systematic assessment of each benchmark’s robustness and reliability.The process includes identifying inadequacies, coding themes within the PPT domains, measuring prevalence, and evaluating benchmark responses.Source: D. Data Analysis · pp. 4–5, Figure 1 · p. 5

IV. OVERVIEW OF LLM BENCHMARKS

  • The surveyed benchmarks were selected under the Unified Evaluation Framework and divided into general-purpose and specialized-domain categories to cover diverse tasks and domains.General-purpose benchmarks target broad knowledge and reasoning, whereas specialized benchmarks target capabilities such as coding, law, medicine, science, tools, and agents.Source: A. LLM Benchmarks and Their… · pp. 5–6

A. LLM Benchmarks and Their Characteristics

  • General-purpose benchmarks use methods including multiple-choice, few-shot and chain-of-thought prompting, human annotation, multi-metric evaluation, and dynamic or semantic testing.Examples span MMLU, Chain-of-Thought Hub, KoLA, ARB, Xiezhi, BIG-bench, AGIEval, HELM, PromptBench, and C-Eval.Source: A. LLM Benchmarks and Their… · p. 5
  • Specialized benchmarks evaluate code synthesis, legal and financial reasoning, medical question answering, scientific problem solving, tool use, API interaction, agent performance, orchestration, and hallucination.They combine automatic metrics, unit tests, human evaluation, rubrics, simulations, comparative analysis, and generated or real-world tools.Source: A. LLM Benchmarks and Their… · p. 6

B. Preliminary Findings Overview

The review found widespread inadequacies in LLM benchmarks, including limited cultural coverage, static interaction formats, and technological sensitivity to superficial prompt changes. It also argues that more comprehensive evaluation practices are needed to assess capabilities and limitations authentically.

B. Preliminary Findings Overview

A. Response Variability in Standardized Evaluations

B. Genuine Reasoning vs Technical Optimization

The paper identifies processual weaknesses that limit benchmark functionality and integrity, spanning reasoning validity, safety trade-offs, linguistic diversity, implementation, and evaluator bias.

B. Genuine Reasoning vs Technical Optimization

  • Benchmarks may mistake memorization or benchmark-specific optimization for genuine reasoning, especially when test items resemble training data.This can overestimate practical utility and cognitive ability, motivating dynamic, adaptable tasks that resist pattern exploitation.Source: B. Genuine Reasoning vs Technical… · pp. 8–9

C. Tension Between Helpfulness and Harmlessness

  • Benchmarks must distinguish useful information from harmlessness, because refusal can appear ethically aligned while avoiding difficult or sensitive questions.Medical, legal, and cybersecurity settings illustrate the need to assess both informative responses and risks such as misinformation or harmful disclosure.Source: C. Tension Between Helpfulness and… · p. 9

D. Linguistic Variability and Embedded Logic Diversity

  • English-centric benchmarks can misrepresent multilingual reasoning and leave uneven moderation or mixed-language jailbreaks undetected.The paper calls for native-language evaluation rather than relying primarily on translation-based testing.Source: D. Linguistic Variability and Embedded… · p. 9

E. Benchmark Installation and Scalability

  • Installation and scaling demands hinder accessible, consistent evaluation, with manual extraction, complex formulations, and large infrastructure requirements creating implementation barriers.Inconsistent deployment can produce environment-dependent results and weaken fair comparisons and resistance to gaming.Source: E. Benchmark Installation and… · p. 10

F. Biases in LLM-Generated LLM Evaluations

  • LLM-generated tasks and judgments can introduce evaluator bias and circularity, allowing models to score through reinforced patterns rather than independent evidence of understanding.The paper recommends integrating human expertise to make evaluations more rigorous and objective.Source: F. Biases in LLM-Generated LLM… · p. 10

VI. PROCESSUAL ELEMENTS

Processual weaknesses make LLM benchmark results difficult to reproduce, compare, and keep current. Inconsistent implementations, slow iteration, and prompt sensitivity can undermine both functionality and integrity.

A. Inconsistent Benchmark Implementation

  • 18/23 benchmarks exhibit inconsistent implementations, including differing few-shot and chain-of-thought setups that produce divergent results.MMLU showed significant score variation across implementation strategies, while M3KE required language-specific customization.Source: A. Inconsistent Benchmark Implementation · pp. 10–11
  • Standardization gaps hinder fair comparison and create opportunities for methodological manipulation or biased interpretations.Uniform implementation protocols are proposed to improve reliability and accurately reflect model capabilities.Source: A. Inconsistent Benchmark Implementation · p. 11

B. Slow Test Iteration Time

  • 18/23 benchmarks suffer slow iteration, with comprehensive frameworks such as BIG-bench and HELM taking weeks or months to evaluate.External coordination, engineering support, and broad scenario coverage contribute to the delay.Source: B. Slow Test Iteration Time · p. 11
  • Delayed evaluations can become outdated as models change and can postpone detection of emerging vulnerabilities.SciBench’s manual extraction and ToolBench’s extensive API configuration illustrate how evaluation complexity extends feedback cycles.Source: B. Slow Test Iteration Time · p. 11

C. Challenge of Proper Prompt Engineering in Benchmarking

  • 14/23 benchmarks face prompt-engineering challenges because formulation differences can substantially alter measured model performance.MMLU’s subject-specific prompts illustrate how inconsistent phrasing may overestimate capabilities in particular areas.Source: C. Challenge of Proper Prompt… · p. 11
  • Leading or ambiguous prompts can misrepresent capabilities, hinder cross-model comparison, and enable optimization for particular prompt styles.The paper recommends prompts that are clear, neutral, and consistent.Source: C. Challenge of Proper Prompt… · p. 11

VII. HUMAN DYNAMICS

Human and societal diversity complicates the consistency, inclusivity, and integrity of LLM benchmarking. Broader discussions also identify benchmark misuse, shallow reasoning assessments, and instability caused by rapidly changing models.

A. Diversity in Human Curators and Evaluators

  • Evaluator differences in cultural, religious, political, and academic backgrounds can introduce subjective inconsistencies into human-judged benchmarks.LegalBench illustrates how differing regional interpretations of legal principles can produce non-uniform assessments.Source: A. Diversity in Human Curators and… · p. 12

B. Diverse Cultural, Social, Political, Religious, and Ideological Norms

  • 18/23 benchmarks struggle to encompass conflicting cultural, religious, political, and ideological norms, making one-size-fits-all evaluation unrealistic.LegalBench may overstate global applicability, while C-Eval can be gamed through context-specific prompts or training data.Source: B. Diverse Cultural, Social,… · p. 12
  • Context-specific benchmarks may be more pragmatic than a universal standard when sensitive responses receive different judgments across societies.The paper frames this as a consequence of inherently subjective human norms rather than evaluator failure.Source: B. Diverse Cultural, Social,… · p. 12

A. Misuse of the Term “Benchmark”

  • Many self-declared LLM benchmarks lack the standardization and comprehensiveness required for genuine benchmarking.The paper distinguishes rigorous, manipulation-resistant evaluations from informal task-specific test sets.Source: A. Misuse of the Term “Benchmark” · pp. 12–13

B. Assessment Limitations for Reasoning and Multimodality

  • Text-focused or simple question-answer benchmarks may conflate recalled knowledge with creative problem-solving and permit dataset-specific optimization.These limitations can misrepresent reasoning, multimodal understanding, adaptability, and real-world capability.Source: B. Assessment Limitations for… · p. 13

C. Unpredictability and Non-Repeatability

  • Frequent model updates make it difficult to determine whether benchmark scores reflect stable capabilities or transient optimizations.This instability can undermine repeatability and allow vendors to target known benchmarks without improving broader performance.Source: C. Unpredictability and… · p. 13

D. Knowledge Boundaries in AI Benchmarking

Benchmark creators’ knowledge limits and linguistic or cultural imbalances constrain comprehensive evaluation. The study therefore proposes broader expertise, staged behavioral profiling, regular audits, and collaborative standards while acknowledging important analytical limitations.

D. Knowledge Boundaries in AI Benchmarking

  • Benchmark creators’ human and domain knowledge can limit assessment of emerging capabilities and weaken integrity in specialized fields such as healthcare or national security.The paper calls for broader knowledge bases and evaluators with deep domain expertise.Source: D. Knowledge Boundaries in AI… · p. 13

E. Challenges in Inclusive Benchmarking

  • English and Simplified Chinese dominance, together with standardized answers, can disadvantage other linguistic and cultural perspectives in reasoning and ethics assessments.The paper highlights BIG-Bench, MultiMedQA, and ARB as examples of narrower coverage.Source: E. Challenges in Inclusive Benchmarking · pp. 13–14

F. Limitations of This Study

  • The critique is limited by subjective bias, uneven benchmark descriptions, non-reproduction of surveyed results, limited language coverage, and absent regulation or public consensus.Only 5 of 23 surveyed studies were peer-reviewed, and evolving models may make conclusions outdated.Source: F. Limitations of This Study · p. 14

G. Future Research Direction: Extending Benchmarks with Behavioral Profiling and Regular Audits

  • Traditional benchmarks can provide an initial competence and compliance screen before deeper behavioral profiling and ongoing assessment.Behavioral profiling targets adaptability, ethical reasoning, and creativity in realistic scenarios, while audits check post-deployment alignment over time.Source: G. Future Research Direction:… · p. 14
  • Dynamic benchmarking with real-time audits and updated standards is proposed to keep evaluation relevant as models and societal needs evolve.The paper calls for collaboration among academia, industry, and regulators to establish standardized methodologies and ethical guidelines.Source: G. Future Research Direction:… · pp. 14–15

IX. CONCLUSION

  • The conclusion advocates behavioral profiling, regular audits, and international collaboration to support robust, trustworthy, and secure LLM deployment.These proposals extend static benchmark scores into more continuous assessment of capabilities and risks.Source: §IX · p. 15

APPENDIX A EXAMPLES OF BENCHMARK INADEQUACIES IN

The surveyed benchmarks often struggle to represent response variability across specialized contexts and to distinguish genuine reasoning from technical optimization. These limitations appear across domain-specific, tool-use, adversarial, and reasoning evaluations.

A. Response Variability in Standardized Evaluations

  • Benchmark design choices such as broad task coverage, fixed prompts, and limited contextual tailoring can overlook specialized model behavior.This concern is reported for MMLU, BIG-Bench, ToolAlpaca, ToolBench, APIBank, C-Eval, and HaluEval.Source: A. Response Variability in… · pp. 17–18
  • Specialized benchmarks often fail to capture model performance accurately across particular contexts, formats, and use cases.Examples include legal reasoning, financial tasks, medical question answering, tool use, and human-centric evaluations.Source: A. Response Variability in… · pp. 17–18
  • External tools and prompting strategies do not uniformly improve performance across scientific problem-solving tasks.SciBench found that varied prompting strategies and external tools failed to enhance models consistently across tasks.Source: A. Response Variability in… · p. 17
  • Adversarial prompts and standardized evaluations expose response variability that can undermine conclusions about models’ real-world robustness.PromptBench found that LLMs were not robust to adversarial prompts, while AgentBench reported substantial performance disparities across commercial and open-source models.Source: A. Response Variability in… · p. 18

B. Genuine Reasoning vs Technical Optimization

  • Final-answer accuracy, functional tests, and benchmark-optimized outputs do not establish genuine reasoning or understanding.SciBench reported incorrectly derived equations, while HumanEval, Chain-of-Thought Hub, AGIEval, and MultiMedQA did not directly resolve the distinction between understanding and optimization.Source: B. Genuine Reasoning vs Technical… · pp. 18–19
  • Rubrics, complex reasoning tasks, and planning methods still leave the source of model performance difficult to determine.ARB, BIG-Bench, C-Eval, ToolBench, and AgentBench evaluate reasoning or planning without definitively separating comprehension from optimization.Source: B. Genuine Reasoning vs Technical… · p. 19

C. Tension Between Helpfulness and Harmlessness

The benchmarks do not provide a resolved methodology for balancing helpfulness with harmlessness. This unresolved tension is especially consequential in domains involving law, morality, medicine, and cybersecurity.

C. Tension Between Helpfulness and Harmlessness

  • Benchmarks lack a clear, resolved criterion for balancing helpfulness and harmlessness across socially consequential tasks.The unresolved challenge is explicitly identified for tasks involving law and morality.Source: C. Tension Between Helpfulness and… · p. 19
  • Human evaluation in medical question answering recognizes the complexity of quantifying helpfulness and harmlessness without standardizing a solution.MultiMedQA relates the difficulty to medical knowledge and societal values.Source: C. Tension Between Helpfulness and… · p. 20
  • Several benchmarks evaluate capability, robustness, or tool use without directly measuring harmful-output risks alongside usefulness.This gap appears in ARB, AGIEval, HELM, ToolBench, PromptBench, C-Eval, LegalBench, and Chain-of-Thought Hub.Source: C. Tension Between Helpfulness and… · p. 20
  • AgentBench does not establish criteria for preventing harmful outputs while evaluating usefulness across environments.Its evaluation setup therefore does not comprehensively cover both functionality and cybersecurity.Source: C. Tension Between Helpfulness and… · p. 20
  • Other benchmark designs similarly focus on hallucination recognition, knowledge assessment, or code correctness rather than the helpfulness–harmlessness trade-off.Examples include HaluEval, M3KE, HumanEval, BOLAA, T-Bench, FLUE, and KoLA.Source: C. Tension Between Helpfulness and… · p. 20

D. Linguistic Variability and Embedded Logic Diversity

Many benchmarks remain linguistically narrow and do not account for the distinct cognitive and logical frameworks embedded in different languages. English-centric, Simplified Chinese, and bilingual designs often leave this diversity unresolved.

D. Linguistic Variability and Embedded Logic Diversity

  • English-centric benchmarks overlook multilingual variation and the cognitive frameworks associated with different languages.This limitation is reported for MMLU, HumanEval, LegalBench, FLUE, MultiMedQA, and HaluEval.Source: D. Linguistic Variability and Embedded… · pp. 21–22
  • Bilingual evaluations may include English and Simplified Chinese without addressing language-specific reasoning structures or embedded logics.The limitation is identified for Chain-of-Thought Hub and AGIEval.Source: D. Linguistic Variability and Embedded… · p. 21
  • Benchmarks focused on Chinese models or educational systems do not clearly distinguish linguistic and logical diversity beyond their selected language setting.This concern applies to Xiezhi, C-Eval, and M3KE.Source: D. Linguistic Variability and Embedded… · pp. 21–22
  • Several tool-use and API benchmarks primarily use English and do not evaluate multilingual capabilities or language-specific logic.ToolAlpaca, ToolBench, T-Bench, and APIBank either omit multilingual evaluation or identify it as future work.Source: D. Linguistic Variability and Embedded… · pp. 21–22
  • HELM acknowledges linguistic diversity as a challenge but provides no solution, with limited consideration of languages beyond English.Its unresolved scope illustrates the broader lack of multilingual benchmark adaptation.Source: D. Linguistic Variability and Embedded… · pp. 21–22

E. Benchmark Installation and Scalability

Benchmark deployment is often difficult to install, scale, and execute consistently, while model-generated evaluation materials can introduce inherited biases. Across these settings, standardized implementation procedures remain largely unresolved.

E. Benchmark Installation and Scalability

  • Scaling evaluations across models and environments often requires complex configuration, adaptation, or additional techniques.This challenge is reported for MMLU, AGIEval, PromptBench, BOLAA, HaluEval, LegalBench, M3KE, T-Bench, KoLA, Xiezhi, and BIG-Bench.Source: E. Benchmark Installation and… · pp. 22–23
  • Installation and scalability require substantial engineering, infrastructure, annotation, or manual grading effort across many benchmarks.Examples include APIBank’s 73 APIs, SciBench’s document processing, and ARB’s manual grading of symbolic and proof-like problems.Source: E. Benchmark Installation and… · p. 22

F. Biases in LLM-Generated LLM Evaluations

  • Generative models used to create tasks, rubrics, labels, training data, or hallucinated samples can transfer their biases and inaccuracies into evaluations.The concern is documented for M3KE, ARB, Xiezhi, BIG-Bench, ToolAlpaca, AgentBench, APIBank, HaluEval, and T-Bench.Source: F. Biases in LLM-Generated LLM… · p. 23

A. Inconsistent Benchmark Implementation

  • Community contributions and ongoing refinement may improve benchmark coverage but do not by themselves ensure uniform execution across research teams.LegalBench and HELM acknowledge implementation challenges without reporting that they are fully resolved.Source: A. Inconsistent Benchmark Implementation · pp. 24–25
  • Benchmark execution remains inconsistent across laboratories because many studies do not standardize procedures for prompts, environments, evaluators, or implementation.This limitation is reported across Chain-of-Thought Hub, KoLA, SciBench, ARB, AGIEval, HumanEval, ToolBench, PromptBench, AgentBench, FLUE, MultiMedQA, and M3KE.Source: A. Inconsistent Benchmark Implementation · p. 24
  • Reliance on a single model for generated evaluation samples can make sample quality depend on that model’s instruction-following capacity.HaluEval explicitly bounds hallucinated-sample quality by ChatGPT’s understanding of the generation instructions.Source: A. Inconsistent Benchmark Implementation · p. 25

B. Slow Test Iteration Time

Slow test iteration time is a recurring inadequacy across many LLM benchmarks, arising from computational expense, manual annotation, complex tasks, and extensive evaluation procedures.

B. Slow Test Iteration Time

  • Several benchmarks, including MMLU, C-EVAL, MultiMedQA, T-Bench, and BOLAA, did not explicitly address slow iteration despite broad or supervised evaluation processes.The supplied assessments generally identify likely delays but report no specific mitigation strategy.Source: B. Slow Test Iteration Time · pp. 25–26
  • Comprehensive benchmarks such as BIG-Bench and HELM require substantial computational resources and lengthy evaluations, without directly addressing iteration speed.BIG-Bench’s programmatic tasks and HELM’s extensive metrics and scenarios make repeated model testing time-consuming.Source: B. Slow Test Iteration Time · p. 26
  • Manual annotation and intervention slow evaluation in AGIEval, ToolAlpaca, APIBank, and HaluEval, while these benchmarks provide no substantial acceleration strategy.HaluEval explicitly links slow iteration to manual labeling and complex instruction-following quality control.Source: B. Slow Test Iteration Time · p. 26
  • PromptBench required iterating over the entire dataset 100 times on average to generate one adversarial prompt, illustrating the tension between thorough testing and rapid iteration.The benchmark did not propose measures to reduce this iteration burden.Source: B. Slow Test Iteration Time · p. 26
  • Complex multi-turn, reasoning, financial, and evolving-task evaluations can prolong testing in AgentBench, Chain-of-Thought Hub, FLUE, KoLA, and SciBench.SciBench additionally relies on extensive manual processing, including LaTeX formatting from PDFs.Source: B. Slow Test Iteration Time · pp. 25–26

C. Challenge of Proper Prompt Engineering

Proper prompt engineering remains a major challenge because prompt design can bias, misinterpret, or incompletely elicit LLM capabilities across benchmark settings.

C. Challenge of Proper Prompt Engineering

  • MMLU, BIG-Bench, and C-EVAL did not adequately address how prompts should elicit capabilities without introducing bias or misinterpretation.These omissions affect assessments spanning broad subjects, reasoning tasks, and multiple disciplines.Source: C. Challenge of Proper Prompt… · pp. 26–27
  • MultiMedQA attempted instruction prompt tuning for medical requirements, yet the difficulty of creating unbiased prompts remained complex and unresolved.The challenge affects both functionality and cybersecurity-related evaluation concerns identified in the assessment.Source: C. Challenge of Proper Prompt… · p. 27
  • ToolBench, AgentBench, BOLAA, and HaluEval incorporated prompts or interaction strategies without fully mitigating biases and limitations in prompt design.Their approaches addressed task interaction or generation procedures, but not the broader adequacy of prompting for capability measurement.Source: C. Challenge of Proper Prompt… · p. 27
  • Reasoning-focused benchmarks such as Chain-of-Thought Hub and ARB acknowledged prompt-engineering difficulty but did not provide conclusive solutions.Both benchmarks seek to assess advanced reasoning, where prompt construction is especially consequential.Source: C. Challenge of Proper Prompt… · p. 27
  • HELM found that model performance is sensitive to prompt formatting and adaptation methods, showing that prompting choices can affect benchmark results.It used standardized few-shot prompting but still reported variation across prompting strategies.Source: C. Challenge of Proper Prompt… · pp. 26–27

APPENDIX C EXAMPLES OF BENCHMARK INADEQUACIES IN HUMAN

Human curators and evaluators introduce potential subjectivity, inconsistency, and bias into benchmark construction and interpretation, while mitigation practices vary across benchmarks.

A. Diversity in Human Curators and Evaluators

  • Many benchmarks, including MMLU, Chain-of-Thought Hub, KoLA, and BIG-Bench, do not adequately address how evaluators’ cultural, academic, political, or religious backgrounds affect results.The assessments connect this diversity to possible subjectivity and inconsistency in benchmark development or evaluation.Source: A. Diversity in Human Curators and… · p. 28
  • MultiMedQA attempted to broaden evaluative perspectives by using clinicians and lay users from diverse locations in its assessment panel.The cited passages identify participants from the USA, UK, and India.Source: A. Diversity in Human Curators and… · p. 28
  • Manual annotation and validation in SciBench, Xiezhi, C-EVAL, APIBank, and HaluEval can introduce variability, but these benchmarks provide limited direct mitigation.SciBench acknowledges diverse interpretation challenges, while APIBank uses multiple-annotator review without specifically resolving background-related bias.Source: A. Diversity in Human Curators and… · pp. 28–29
  • AGIEval and LegalBench recognize human-centered or collaborative evaluation contexts but do not detail measures for resolving diversity-related inconsistency.LegalBench’s tasks came from legal professionals, while AGIEval relies on high-standard human-centric examinations.Source: A. Diversity in Human Curators and… · p. 28
  • Human involvement in ToolAlpaca, ToolBench, T-Bench, AgentBench, and PromptBench is not consistently accompanied by strategies to control evaluator-related bias or interpretive variation.ToolAlpaca sought a diverse corpus, but the assessment distinguishes this from addressing evaluator diversity itself.Source: A. Diversity in Human Curators and… · pp. 28–29

B. Diverse Cultural, Social, Political, Religious and Ideological Norms

Many benchmarks rely on standardized answers, rubrics, or interpretations without fully incorporating diverse cultural, social, political, religious, and ideological norms.

B. Diverse Cultural, Social, Political, Religious and Ideological Norms

  • MMLU, Chain-of-Thought Hub, KoLA, and SciBench may not capture pluralistic values because their evaluation criteria do not explicitly integrate cultural or ideological diversity.This concern is especially relevant to ethics, societal norms, and other interpretation-sensitive tasks.Source: B. Diverse Cultural, Social,… · pp. 29–30
  • LegalBench, FLUE, MultiMedQA, and M3KE acknowledge or encounter domain-specific diversity challenges but do not provide resolved methods for reconciling divergent norms.The unresolved issues include cross-jurisdictional legal standards, culturally sensitive financial tasks, clinical applications, and varied subject domains.Source: B. Diverse Cultural, Social,… · p. 29
  • Xiezhi attempted greater cultural balance through less China-centric question subsets, but the broader challenge of integrating diverse norms remained unresolved.The adjustment aimed at a more culturally inclusive assessment without fully spanning ideological and cultural variation.Source: B. Diverse Cultural, Social,… · p. 30
  • BIG-Bench, PromptBench, AgentBench, APIBank, and HaluEval use standardized evaluation structures without adequately addressing varied cultural and ideological perspectives.The assessment associates this omission with potentially biased or nonrepresentative results.Source: B. Diverse Cultural, Social,… · pp. 30–31
  • AGIEval and C-EVAL incorporated multiple languages, disciplines, or context-specific coverage, yet neither fully resolved the limitations of standardized answers and rubrics.AGIEval included English and Simplified Chinese exams, while C-EVAL emphasized diverse disciplines and difficulty levels in a Simplified Chinese context.Source: B. Diverse Cultural, Social,… · pp. 29–30
  • HELM measured fairness, bias, and toxicity across languages and cultural contexts but acknowledged difficulty standardizing benchmarks that represent pluralistic human beliefs and values.Its broad coverage therefore did not eliminate the underlying representational challenge.Source: B. Diverse Cultural, Social,… · p. 30
Loading 2402.09880v2…