PaperlayerView exact revision on arXiv

Source-linked AI summary

Dynamic Evaluation of Large Language Models by Meta Probing Agents

Kaijie Zhu, Jindong Wang, Qinlin Zhao, Ruochen Xu, Xing Xie

arXiv:2402.14865v2cs.CLcs.AIcs.LG

TL;DR

Public LLM evaluations face concerns about data contamination and often offer limited analysis of abilities across tasks. MPA uses psychometrically inspired probing and judging agents to create configurable evaluation questions; most LLMs perform worse on its dynamic benchmarks, while the analysis reports ability correlations and stronger correlations among larger models.

  • Problem

    Public benchmarks raise data-contamination concerns, and existing evaluations often provide overall scores rather than fine-grained analysis of how LLM abilities relate.

    Source: §1 · p. 1

  • Method

    MPA uses probing and judge agents to transform existing evaluation problems according to configurable principles for language understanding, problem solving, and domain knowledge.

    Source: §1 · p. 2

  • Results

    Most LLMs performed worse on dynamic probing benchmarks, while language understanding and problem solving were most strongly correlated and larger models tended to have stronger ability correlations.

    Source: §1 · p. 2

  • Takeaways & Limitations

    MPA supports multifaceted analysis of LLM abilities rather than evaluation limited to overall benchmark results.

    Source: §1 · p. 2, §5 · p. 7

    Some generated questions deviated from their original intent despite the judge agent’s validity and consistency checks.

    Source: §7 · p. 9

Abstract

from arXiv · show

Evaluation of large language models (LLMs) has raised great concerns in the community due to the issue of data contamination. Existing work designed evaluation protocols using well-defined algorithms for specific tasks, which cannot be easily extended to diverse scenarios. Moreover, current evaluation benchmarks can only provide the overall benchmark results and cannot support a fine-grained and multifaceted analysis of LLMs' abilities. In this paper, we propose meta probing agents (MPA), a general dynamic evaluation protocol inspired by psychometrics to evaluate LLMs. MPA is the key component of DyVal 2, which naturally extends the previous DyVal~\citep{zhu2023dyval}. MPA designs the probing and judging agents to automatically transform an original evaluation problem into a new one following psychometric theory on three basic cognitive abilities: language understanding, problem solving, and domain knowledge. These basic abilities are also dynamically configurable, allowing multifaceted analysis. We conducted extensive evaluations using MPA and found that most LLMs achieve poorer performance, indicating room for improvement. Our multifaceted analysis demonstrated the strong correlation between the basic abilities and an implicit Matthew effect on model size, i.e., larger models possess stronger correlations of the abilities. MPA can also be used as a data augmentation approach to enhance LLMs. Code is available at: https://github.com/microsoft/promptbench.

1. Introduction

Public benchmarks raise concerns about data contamination and often provide limited insight into the relationships among LLM abilities. MPA uses psychometrically inspired agents to generate flexible probing benchmarks; evaluations found lower performance on these benchmarks and correlations among abilities.

1. Introduction

  • LLMs performed significantly worse on MPA probing benchmarks than on original benchmarks, suggesting potential data contamination in current benchmarks.Figure 1 compares vanilla MMLU with probing sets for language understanding, problem solving, and domain knowledge.Source: Figure 1 · p. 1, §1 · p. 2
  • Existing dynamic evaluations focused on specific tasks, while benchmark results often provide limited insight into how abilities relate within a task.For example, a math application problem requires understanding the question and solving it.Source: §1 · pp. 1–2
  • MPA transforms existing problems into new evaluation questions with probing and judge agents, guided by principles for language understanding, problem solving, and domain knowledge.The principles can be combined flexibly for multifaceted ability analysis, and the judge agent checks generated samples for consistency and relevance.Source: §1 · p. 2
  • Language understanding and problem solving have the strongest correlation among the three basic abilities, and larger models tend to show stronger correlations.The paper also reports varied error patterns in its fine-grained analysis.Source: §1 · p. 2
  • MPA can also serve as a data-augmentation approach to improve LLM performance.The paper presents this as an additional use of the protocol.Source: §1 · p. 2

2. Related Work

Prior work includes diverse benchmark designs, contamination-detection methods, and dynamic evaluation strategies, but many dynamic methods target specific tasks. MPA differs through psychometrically inspired sample generation and support for multifaceted analysis.

LLMs Evaluation & Data Contamination

  • Evaluation approaches include human-centric feedback, crowdsourced tests, custom challenging tests, and specialized-task benchmarks.Examples include AdaVision, DynaBench, HELM, and Big-Bench.Source: §2 · p. 2
  • MPA’s principles can be flexibly combined to form probing benchmarks for multifaceted analysis.Figure 2 illustrates the principles and shows a new sample generated from an ARC-C example.Source: Figure 2 · p. 3
  • Dynamic strategies were proposed to reduce data contamination, but MPA differs in its sample-generation mechanism and support for multifaceted analysis.The related work describes DyVal and NPHardEval as dynamic evaluation approaches.Source: §2 · pp. 2–3

LLMs as Autonomous Agents

  • LLM-agent research includes autonomous task-completion agents, LLM judges, and systems that enhance training data or align outputs.The paper distinguishes its work by using psychometrically inspired principles for sample generation and multifaceted analysis.Source: §2 · p. 3

3. Method

MPA uses psychometric principles and probing and judging agents to dynamically transform benchmark questions while checking their consistency. Its framework evaluates language understanding, problem solving, and domain knowledge, with principles that can be combined for multifaceted analysis.

3.1. Overview

  • MPA uses LLM agents to generate new evaluation problems from benchmark samples according to psychometric principles, with a judge checking consistency and a dynamic combination of principles enabling multifaceted analysis.The probing agent retries when the judge rejects a question; accepted questions pass the consistency check.Source: §3.1 · pp. 3–4

3.2. Probing Agent

  • The probing agent transforms an original question according to a principle while aiming to preserve its core meaning and correctness.The prompts direct it to create semantically similar but structurally different questions.Source: §3.2 · p. 4

3.3. Judge Agent

  • The judge agent uses a binary Yes/No verdict to assess whether a generated question preserves the original question’s intent and informational content.Human evaluation involved 1,800 questions and reported 94% overall accuracy and 97% for each question.Source: §3.3 · p. 4

3.4. Psychometric principles

  • The framework organizes five probing principles around three psychometric abilities: language understanding, problem solving, and domain knowledge.Table 1 presents prompts for the principles based on these abilities.Source: Table 1 · p. 4, §3.4 · p. 5

3.4.1. LANGUAGE UNDERSTANDING

  • Language understanding is probed through question paraphrasing, choice paraphrasing, and choice permutation, which tests sensitivity to answer position.Choice-related principles do not apply to non-QA benchmarks such as GSM8K.Source: §3.4.1 · p. 5, §3.4.2 · p. 5

3.4.2. PROBLEM SOLVING

  • Problem solving is tested by adding relevant but non-essential context, assessing whether a model filters extraneous information and focuses on what is needed to solve the question.The authors avoid generating completely new problems because their correctness cannot be guaranteed.Source: §3.4.2 · p. 5

3.4.3. DOMAIN KNOWLEDGE

  • Domain knowledge concerns the depth and accuracy of specific-area knowledge; adding a relevant but incorrect answer choice tests whether the model can exclude it.The framework is not limited to the five principles described and allows more to be added.Source: §3.4.3 · p. 5

4. Experiments

The experiments test MPA across multiple benchmarks and LLMs, finding performance declines on dynamically generated questions and examining probing principles, prompting, contamination, and agent reliability.

4.1. Experimental Setup

  • The evaluation covered MMLU, ARC-C, GSM8K, and BBH, with six LLMs evaluated and GPT-4-Turbo serving as the main probing and judging agent.Evaluated models generally used temperature 0 and 1,000-token generation; the authors noted that only GPT-4 was then capable of generating and judging questions.Source: §4.1 · pp. 5–6

4.3. Effect of Different Probing Principles

  • Principles 1, 2, and 5 were most effective, while principle 3 was less effective and could increase GPT-4-Turbo and GPT-3.5-Turbo accuracy on ARC-C.Source: Figure 4 · p. 6, §4.3 · p. 6

4.2. Main Results

  • 15.7% worse on MMLU than on the original benchmark: all LLMs declined on MPA benchmarks, MMLU/ARC-C drops exceeded GSM8K/BBH, and GPT-4-Turbo remained strongest.The confusion-matrix analysis found frequent OT/PF cases and more such instances for open-source than proprietary models, suggesting possible greater susceptibility to contamination.Source: §4.2 · p. 6, Figure 3 · p. 6, Table 2 · p. 7

4.4. Ablation Study on Prompt Engineering

  • Neither CoT nor ICL reliably boosted probing-benchmark performance; effects varied by model and dataset.GPT-4-Turbo improved from 88.50 to 89.39 on GSM8K with ICL, whereas GPT-3.5-Turbo declined under the same conditions.Source: Table 3 · p. 7, §4.4 · p. 7

4.5. Albation Study on Data Contamination

  • On 30 expert-written questions, GPT-4 scored 60% on both original and probing items, with a 3% OT/PF ratio, below common public benchmarks.Source: §4.5 · p. 7

4.6. Weak LLMs as Probing and Judge Agents

  • Weaker judges missed meaning-changing probes, while weaker probing agents altered question essence and GPT-4-Turbo did not consistently catch these shifts.Source: §4.6 · p. 7

5. Multifaceted Analysis of the Basic Abilities

This section uses MPA for multifaceted analysis of benchmark complexity, relationships among three basic abilities, model size, and error patterns.

5.1. Analysis on Benchmark Complexity

  • As benchmark complexity increases, all evaluated LLMs perform worse, while GPT-4-Turbo consistently achieves the highest precision.Complexity is increased by combining principles for language understanding, problem solving, and domain knowledge.Source: §5.1 · p. 7

5.2. Relationship of the Basic Abilities

  • All three abilities are highly correlated, with language understanding and problem solving more closely related than the other ability pairs.The authors suggest these two abilities may predict each other.Source: Table 4 · p. 8, §5.2 · p. 8

5.3. Analysis on Model Size

  • Larger models tend to show stronger correlations between basic abilities, an association the authors describe as an implicit Matthew Effect.For Llama2 variants, each ability positively correlates with model size at nearly the same slope.Source: Figure 6 · p. 8, §5.3 · p. 8

5.4. Error Analysis

  • In 50 cases where GPT-4-Turbo got original GSM8K questions right but transformed versions wrong, errors included misunderstanding intent, missing required formats, and calculation mistakes.The language-understanding errors distinguish question misunderstanding from instruction-following failures.Source: §5.4 · p. 8
  • Across 57 MMLU tasks, professional law had the highest topic error rate, followed by moral scenarios and professional psychology.The authors characterize these as predominantly professional and ethical tasks.Source: §5.4 · p. 8
  • Performance degradation may reflect ambiguity in original questions or transformed questions that diverge in meaning from their originals.The authors note that GPT-4-Turbo often gave plausible answers to ambiguous originals, suggesting potential data-memorization issues.Source: §5.4 · p. 8

6. MPA as Data Augmentation for LLMs

The authors pilot MPA-generated data as fine-tuning augmentation for GPT-3.5-Turbo. Fine-tuning improved results on both MMLU and ARC-C.

6. MPA as Data Augmentation for LLMs

  • The pilot used MMLU and ARC-C training splits to generate probing samples, then evaluated the fine-tuned model on original test splits and probing benchmarks.Fine-tuning data combined questions generated separately by five principles with the original training set.Source: §6 · pp. 8–9
  • 2% average improvement on both MMLU and ARC-C followed fine-tuning GPT-3.5-Turbo with MPA-generated data.Figure 8 compares GPT-3.5-Turbo and its fine-tuned version on the original benchmarks and probing benchmarks.Source: Figure 8 · p. 9, §6 · p. 9
  • The authors present the results as evidence that MPA-generated samples can serve as data augmentation as well as evaluation material.The reported improvements were observed in this pilot fine-tuning study.Source: §6 · p. 9

7. Conclusion and Discussion

The conclusion presents MPA as a dynamic evaluation protocol for analyzing three LLM abilities and as a source of training data. It also identifies limits in dataset breadth and probing-benchmark validity.

7. Conclusion and Discussion

  • MPA uses dynamic evaluation to analyze three cognitive abilities of LLMs, and its generated samples can also be used to improve training as data augmentation.The conclusion frames the protocol as inspired by psychometric theory.Source: §7 · p. 9
  • The study covers only four datasets and a specific range of topics, limiting the breadth of its evaluation evidence.Source: §7 · p. 9
  • Some probing questions deviated from their original intent despite the judge agent's checks, leaving room to improve MPA's robustness and effectiveness.Source: §7 · p. 9

Impact Statement

The impact statement presents evaluating general LLM abilities as important for responsible AI and for understanding models, and says the agents were controlled to avoid harmful content.

Impact Statement

  • Evaluating general LLM abilities is framed as essential to responsible AI for society.Source: Impact Statement · p. 9
  • The proposed evaluation protocol aims to measure models' true capabilities and foster better understanding of them.Source: Impact Statement · p. 9
  • The authors say they carefully controlled the generative agents to ensure they would not generate harmful content.Source: Impact Statement · p. 9

A. Datasets

The appendices describe the evaluation datasets and prompts, then report that results were averaged over three runs because the dynamic protocol introduces randomness.

A. Datasets

  • MMLU covers 13,985 test samples across 57 tasks; ARC-C has 1,172 science questions, GSM8K has 1,319 math problems, and three BBH subsets each have 250 samples.The BBH subsets are Formal Fallacies, Object Counting, and Temporal Sequences.Source: A. Datasets · p. 13

B. Evaluation Prompts

  • Evaluation prompts introduce the task or problem and ask models to place their answer inside <<<>>>.The examples include multiple-choice, math, and science prompts.Source: B. Evaluation Prompts · p. 13

BBH (formal fallaices)

  • The Formal Fallacies prompt frames the task as judging whether an argument follows logically from a set of statements.The answer format illustrates responses such as “valid.”Source: BBH (formal fallaices) · p. 13

BBH (object counting)

  • The Object Counting prompt asks for the number of items in a specified class among a collection of possessions.The answer is requested inside <<<>>>.Source: BBH (object counting) · pp. 13–14

BBH (temporal sequences)

  • The Temporal Sequences prompt asks when a person could have performed an activity given a sequence of events.The response format illustrates an answer in parentheses inside <<<>>>.Source: BBH (temporal sequences) · p. 14

C.1. Standard Deviation of Main Results

  • Because the protocol introduces randomness, experiments were run three times and averaged; reported standard deviations were small across models and datasets.Table 5 reports the standard deviation of the main results.Source: §C.1 · p. 14, Table 5 · p. 14

C.2. Results of Different Modular Principles

The appendix presents results for different modular principles and abilities, and examines GPT-4-Turbo’s most frequent MMLU error topics.

C.2. Results of Different Modular Principles

  • Tables 6 and 7 report results for different principles on MMLU, ARC-Challenge, GSM8K, and partial BBH samples.The appendix notes that only partial BBH samples were adopted; the supplied passages do not give table values.Source: §C.2 · p. 14, Table 6 · p. 14, Table 7 · p. 14

C.3. Results of Relationship of the Basic Abilities

  • Table 8 presents results of different abilities, while its caption describes results for different LLMs on ARC-C and MMLU.The supplied passages do not report the table’s cell values.Source: §C.3 · p. 14, Table 8 · p. 15

C.4. Top topics of MMLU

  • GPT-4-Turbo made the most MMLU errors in topics including professional law, moral scenarios, and security studies, with ethics and psychology also identified as areas of potential weakness.The passage suggests insufficient training data and ambiguous ground truth as possible reasons for some errors.Source: §C.4 · pp. 14–15, Figure 9 · p. 15

C.5. Examples of wrong/ambiguous evaluation samples

The appendix gives examples of ambiguous evaluation items and of MPA-generated questions across several benchmark task types.

C.5. Examples of wrong/ambiguous evaluation samples

  • For a hurdle-time question, Lee runs 38 seconds, two seconds faster than Gerald before his diet change improved speed by 10%; Gerald’s improved time is 36.3636 seconds.Source: §C.5 · p. 15
  • A GSM8K interest question leaves simple versus compound interest unspecified, making its answer ambiguous; the provided $106 answer implies simple interest.The analysis contrasts this with the possibility of compound interest.Source: §C.5 · p. 15

D. Examples generated by MPA

  • MPA’s generated examples reframe questions about Roman religious diversity and Janet’s egg sales as longer, contextualized prompts.The examples retain the underlying subject matter while adding narrative context.Source: D. Examples generated by MPA · pp. 16–17
  • Other generated examples recast planetary rotation and formal fallacy tasks with expanded scenarios and wording.The formal fallacy example centers on whether a conclusion follows from the stated premise.Source: D. Examples generated by MPA · p. 17
  • The appendix also illustrates MPA prompts for counting musical instruments and identifying a possible coffee-shop interval from a daily schedule.The temporal-sequence example supplies the person’s activities and the shop’s closing time.Source: D. Examples generated by MPA · p. 18

E. Human verification results

Table 9 is identified as reporting human verification results on MMLU and ARC-Challenge.

E. Human verification results

  • Table 9 reports human verification results on MMLU and ARC-Challenge; the supplied passage does not state the results.Source: Table 9 · p. 19
Loading 2402.14865v2…