PaperlayerView exact revision on arXiv

Source-linked AI summary

Benchmarking Foundation Models with Language-Model-as-an-Examiner

Yushi Bai, Jiahao Ying, Yixin Cao, Xin Lv, Yuze He, Xiaozhi Wang, Jifan Yu, Kaisheng Zeng, Yijia Xiao, Haozhe Lyu, Jiayin Zhang, Juanzi Li, Lei Hou

arXiv:2306.04181v2cs.CLcs.LG

TL;DR

Existing open-ended QA benchmarks face testing leakage and difficulties automating evaluation. Language-Model-as-an-Examiner uses knowledgeable LMs to generate and assess questions across breadth and depth, with scoring, ranking, and peer-examination. The framework benchmarks foundation models and reports more diverse questions and fairer evaluation outcomes through peer-examination.

  • Problem

    Existing benchmarking pipelines face testing leakage and difficulty evaluating open-ended answers with automated metrics.

  • Method

    The framework uses an LM as a knowledgeable examiner that generates diverse and follow-up questions, scores and ranks responses, and supports decentralized peer-examination.

  • Results

    Peer-examination generates more diverse questions and balances individual evaluator biases, producing a more equitable evaluation outcome across benchmarks of eight foundation models.

  • Takeaways & Limitations

    LM-generated questions, multi-round probing, combined scoring and ranking, and peer examination provide a benchmark designed for broader and fairer foundation-model assessment.

  • Takeaways & Limitations

    Evaluation may reflect model-specific linguistic, radical, or gender biases, and only four accessible models currently support large-scale peer-examination.

Abstract

from arXiv · show

Numerous benchmarks have been established to assess the performance of foundation models on open-ended question answering, which serves as a comprehensive test of a model's ability to understand and generate language in a manner similar to humans. Most of these works focus on proposing new datasets, however, we see two main issues within previous benchmarking pipelines, namely testing leakage and evaluation automation. In this paper, we propose a novel benchmarking framework, Language-Model-as-an-Examiner, where the LM serves as a knowledgeable examiner that formulates questions based on its knowledge and evaluates responses in a reference-free manner. Our framework allows for effortless extensibility as various LMs can be adopted as the examiner, and the questions can be constantly updated given more diverse trigger topics. For a more comprehensive and equitable evaluation, we devise three strategies: (1) We instruct the LM examiner to generate questions across a multitude of domains to probe for a broad acquisition, and raise follow-up questions to engage in a more in-depth assessment. (2) Upon evaluation, the examiner combines both scoring and ranking measurements, providing a reliable result as it aligns closely with human annotations. (3) We additionally propose a decentralized Peer-examination method to address the biases in a single examiner. Our data and benchmarking results are available at: http://lmexam.xlore.cn.

1 Introduction

The paper identifies testing leakage and automated evaluation as key limitations of open-ended QA benchmarking, then proposes Language-Model-as-an-Examiner with breadth, depth, scoring, ranking, and peer-examination strategies.

  • Existing benchmarks risk overestimating performance when test answers overlap with foundation-model pre-training data.
  • Automated metrics struggle with open-ended QA because many valid answers make ground-truth similarity ineffective.
  • A pilot study found 18 of 100 GPT-4 judgments of LLaMA answers on Natural Questions were incorrect.
  • Language-Model-as-an-Examiner uses an LM to formulate questions from inherent knowledge and evaluate other models’ responses.
  • The framework probes knowledge breadth and depth, combines Likert scoring with ranking, and uses peer-examination to reduce single-examiner bias.
  • Experiments benchmark eight foundation models and report that peer-examination produces more diverse questions and more equitable evaluation outcomes.

2 Related Work

Related work covers open-ended QA benchmarks and automated NLG evaluation, highlighting scalability limits, metric shortcomings, and the growing use of language models for reference-free assessment.

  • Benchmarks for Foundation Models: Existing open-ended QA benchmarks include MS MARCO, SQuAD, Natural Questions, WebQuestions, OpenBookQA, and the long-form QA dataset ELI5.
  • Benchmarks for Foundation Models: Human curation and annotation constrain benchmark scalability, whereas LM-based dataset construction offers effortless extensibility.
  • Automating NLG Evaluation: Automatic metrics such as F1, Exact Match, BLEU, ROUGE, and METEOR can produce large discrepancies between tested and actual performance.
  • Automating NLG Evaluation: Reference-based LM metrics measure distributional similarity rather than lexical overlap, supporting more flexible generation contexts.

3 Methodology

The methodology constructs LMExamQA across domains and cognitive levels, adds follow-up questioning for depth, evaluates responses with scoring and ranking, and aggregates decentralized peer judgments.

  • 3.1 Dataset Construction: LMExamQA samples domains from Google Trends Categories and prompts an LM to generate distinct questions with varied forms and cognitive levels.
  • 3.1 Dataset Construction: The dataset distinguishes memorization, comprehension, and analysis questions, with analysis requiring comparison, impact, or advantages-and-disadvantages reasoning.
  • 3.1 Dataset Construction: LMExamQA has a more balanced distribution across three cognitive levels than previous datasets and contains a larger proportion of higher-level questions, particularly analysis questions.
  • 3.1 Dataset Construction: Human experts found the LM examiner’s generated answers 100% accurate on a random sample of 100 questions.
  • 3.1 Dataset Construction: Multi-round follow-up questions are conditioned on preceding answers to probe deeper comprehension of a topic.
  • 3.2 Evaluation Metrics: The benchmark combines absolute Likert scoring across predefined dimensions with relative ranking of responses.
  • 3.2 Evaluation Metrics: Ranking n contestant models uses merge sort with O(n log n) pairwise comparisons.
  • 3.3 Peer-Examination: Peer-examination assigns each participating model to examine others, then combines their evaluations by voting to use collective expertise and diverse perspectives.

4 Experiments

Experiments evaluate the framework’s scoring, ranking, breadth, depth, and peer-examination strategies across foundation models. Results show differentiated model performance, deeper probing effects, evaluator bias, and potential benefits from decentralized examination.

  • Experimental Setup: GPT-4 serves as the centralized examiner, while peer examination uses Claude, ChatGPT, Bard, and Vicuna-13B as examiners.The centralized setup evaluates eight foundation models on LMExamQA; peer examination assigns each of four models 100 questions across 20 domains.
  • Centralized Benchmarking Results: 5-shot examples improve performance by 17%, 123%, and 206% on memorization, comprehension, and analysis questions, respectively, for models without SFT.The largest relative gain occurs on analysis questions, which require more sophisticated answers.
  • Centralized Benchmarking Results: All models score over 80/100 on factuality and coherence, but they vary widely in comprehensiveness.Comprehensiveness measures whether responses address all aspects of a question.
  • Centralized Benchmarking Results: Ranking distinguishes model quality more clearly than scores, with ChatGPT achieving a 68% win rate over Vicuna-13B.The authors report that fine-tuned models have near-perfect scores, whereas pairwise ranking reveals a notable difference between the two models.
  • Peer-Examination Results: Peer examination produces diverse question distributions and combines differing evaluator preferences to balance individual biases and yield a more equitable outcome.Questions generated by different models exhibit distinct distributions within domain clusters, while evaluator preferences differ in the paraphrase-bias experiment.

5 Conclusion

The paper introduces Language-Model-as-an-Examiner and LMExamQA for comprehensive knowledge probing, benchmarking foundation models, and fairer evaluation through peer examination.

  • Language-Model-as-an-Examiner uses a language model to generate questions and evaluate foundation-model responses.
  • LMExamQA is designed to probe knowledge comprehensively and across stratified domains.
  • Peer examination mitigates potential biases associated with relying on a single examiner.
  • The framework could be expanded with domain-specific language models or vision-language models for more holistic evaluation.

6 Limitation

The authors identify evaluation bias and limited peer-examination capability as key limitations, while also noting privacy, harmful-content, and fairness considerations for LM-generated data.

  • Evaluation results may reflect models’ preferences for linguistic styles and other biases, including radical and gender bias.
  • Large-scale peer examination is constrained because only four accessible models demonstrate the required machine-generated-text assessment capability.
  • The authors report no observed personal-information leakage in LMExamQA and no misinformation, hate speech, or harmful content after review.
  • The dataset’s fairness remains a consideration because language models can inherit and amplify societal biases from training data.

A Case Study in Pilot Study

The pilot study shows that GPT-4 can misjudge both correct and incorrect LLaMA-13B answers, with both errors attributed to inadequate knowledge of the questions.

  • GPT-4 incorrectly marked a correct LLaMA-13B answer about when $2 bills stopped being made as false.
  • GPT-4 erroneously marked LLaMA-13B’s incorrect answer about the first earthquake early warning system as accurate.
  • Both GPT-4 misjudgments stemmed from inadequate understanding of knowledge related to the original questions.

B Example Prompts

The example prompts operationalize broad and deep question generation, multi-dimensional scoring, pairwise ranking, rewriting, and domain-level performance analysis.

  • B.1 Question Generation: Question-generation prompts require domain coverage across breadth and depth, using varied question words and concise initial knowledge questions.
  • B.2 Multi-round Question Generation: Follow-up prompts generate deeper questions from a previous question-and-answer pair and require confidence in the proposed question.
  • B.3 Peer-Examination Question Generation: Peer-examination prompts request five complex questions that are difficult for prominent AI systems while remaining answerable by the writer.
  • B.4 Likert Scale Scoring: Likert scoring rates accuracy, coherence, factuality, and comprehensiveness from 1 to 3, plus an overall score from 1 to 5.
  • B.5 Pairwise Evaluation: Pairwise evaluation asks the examiner to select the better of two responses using accuracy, coherence, factuality, and comprehensiveness.
  • B.6 ChatGPT Rewrite: Rewrite prompts require eloquent paraphrasing that preserves all original points and details without adding knowledge.
  • B.6 ChatGPT Rewrite: Figure 8 reports percentages of full-mark answers across first-level domains using 5-shot performance for models without SFT.

C.1 Metric Evaluation

The metric evaluation compares absolute scoring with ranking-based judgments of model-generated responses. Human annotators score responses on a 1–5 overall scale and rank three responses per question.

  • Human annotators evaluated 300 model-generated responses from 100 LMExamQA questions using scores and rankings.Three responses were randomly selected for each question.
  • Each response received a 1–5 overall score based on accuracy, coherence, factuality, and comprehensiveness.The first four dimensions use 1–3 ratings, while the overall score ranges from 1 to 5.
  • Annotators also ranked the three responses for each question by holistic answer quality.Ties could be represented with “=” when responses were nearly indistinguishable.

C.2 Benchmarking

The benchmark evaluates foundation models with single-round and multi-round questions, while examining memorization, knowledge depth, and evaluator consistency. Its examples show that follow-up questioning can expose weaknesses not visible in initial answers.

  • C.2 Benchmarking: Single-round benchmarking uses standardized prompts to evaluate foundation models across LMExamQA domains.The benchmark provides 0-shot and 5-shot prompts, and uses a 200-token maximum output with temperature 0.
  • C.2 Benchmarking: Multi-round questioning tests whether models can answer increasingly sophisticated follow-ups based on preceding responses.This setting is intended to probe knowledge more deeply than isolated questions.
  • C.2 Benchmarking: Models’ accuracy remained close to first-round performance despite answering the initial questions correctly.The authors relate this pattern to foundation models’ non-localized knowledge structure compared with humans.
  • C.2 Benchmarking: Single-round performance may produce deceptive QA capability estimates because models can recall seen text without genuine understanding or analysis.This is presented as a potential limitation of relying solely on single-round evaluation.
  • C.2 Benchmarking: Peer-examination uses multiple examiners after testing comparison consistency under reversed response order.Bard, Claude, and ChatGPT were selected as examiners, whereas Vicuna-13B was excluded for inconsistent comparisons.
  • C.2 Benchmarking: ChatGPT evaluations use a 400-token response limit to reduce forgetting, while potentially removing important details or context.The authors explicitly trade some fairness for more reliable ChatGPT comparisons.
  • C.2 Benchmarking: In a cosmetic-surgery example, ChatGPT received an overall score of 5, while Flan-UL2 received 4 and BLOOMZ received 3.The scores reflect differences in answer accuracy, factuality, coherence, and comprehensiveness.
  • C.2 Benchmarking: A health-policy example shows ChatGPT receiving an overall score of 5 after giving a comprehensive answer about pandemic-driven policy changes.The evaluation includes dimension-level scores and written reasons.

D.1.3 Knowledge Analysis Question Case

The bundling-services case evaluates whether answers address both benefits and drawbacks in pricing and service quality. The scoring emphasizes correctness and clarity but penalizes incomplete explanation.

  • The question asks for advantages and disadvantages of bundling internet, television, and phone services from one provider.The stated advantages include cost savings and convenience, while disadvantages include limited options.
  • The evaluated answer was judged accurate, coherent, and factual but insufficiently comprehensive.Its general overview did not fully elaborate the reasons behind cost savings or potential issues.

D.2 Multi-turn Question Evaluation

Multi-turn evaluation reveals that models often provide partially correct follow-up answers while failing to address comparisons, disadvantages, or important distinctions. The examples illustrate how follow-up questions test depth beyond basic recall.

  • Flan-T5 correctly identifies aluminium as a common road-bike-frame material but omits disadvantages and comparisons with carbon fiber and steel.Its follow-up response discusses aluminium’s advantages, including low cost and ease of welding.
  • The aluminium follow-up answer received an overall score of 2 because it provided accurate information but did not fully address the question.Its comprehensiveness score was 1.
  • GLM-130B’s basketball follow-up answer received an overall score of 3 because it described defensive strategies without sufficient detail.The response incorrectly characterized trapping as one defender blocking the ball-handler, rather than typically involving two defenders closing in.
  • The basketball example shows that correct identification of strategies does not ensure accurate explanation of their distinctions.The answer covered trapping, double-teaming, and switching but lacked important details.
  • LLaMA-65B answers a global-health question and receives a follow-up asking how the World Health Organization develops and implements its policies.The example directly extends a factual question into a process-oriented assessment.

D.3 Peer-Examination Evaluation

The peer-examination examples show foundation models answering open-ended questions across psychology and artificial intelligence. Responses address both explanatory topics and long-horizon predictions, combining concepts, examples, and expected developments.

  • The psychology response links attachment styles to relationship patterns, distinguishing secure, anxious, and avoidant styles.It associates insecure styles with jealousy, possessiveness, trust problems, dependency, fear of abandonment, or emotional distance.
  • The AI forecast identifies hardware, data availability, and algorithmic development as drivers of progress over the next 10–20 years.
  • The forecast highlights reinforcement learning, transfer learning, and generative models as approaches that may substantially reshape AI research and development.It connects these approaches with more sophisticated decision-making, faster learning, and improved performance across applications.
Loading 2306.04181v2…