PertEval: Unveiling Real Knowledge Capacity of LLMs with Knowledge-Invariant Perturbations

Motivation

• Static benchmarks can mismeasure knowledge because fixed close-ended formats limit scenarios and published test data can be memorized during training.These issues motivate testing across varied scenarios while retaining expert-designed benchmark knowledge.Source: §1 · p. 2

PertEval

• PertEval generates knowledge-invariant variants of static benchmarks through content-level and format-level perturbations, then compares responses across versions.Its response consistency analyses measure capacity and investigate performance changes.Source: §1 · pp. 2–3, Figure 1 · p. 2

Results

• 25.8% absolute overestimation for GPT-4 and 38.8% for Gemini-1.0-Pro on MMLU show that static benchmarks substantially overestimate knowledge capacity.The evaluation re-examines six representative LLMs.Source: §1 · p. 2
• Performance declines are mainly associated with selecting extra incorrect choices, increasing by up to 9.7% for GPT-4 and 27.7% for Gemini-1.0-Pro.This response-pattern result helps explain the gap between raw and perturbed evaluations.Source: §1 · p. 2

Implications

• Response analyses reveal uncertainty about specious knowledge, potential rote memorization of correct options, and vulnerability to content-level and global-order perturbations.The findings identify weaknesses that can inform refinement of LLM knowledge mastery.Source: §1 · p. 3

Knowledge Capacity Evaluation

• Knowledge-capacity evaluation research covers general and professional multiple-choice benchmarks plus data- and model-driven methods for improving evaluation accuracy.Existing methods include contamination detection, new test generation, psychometric techniques, dynamic evaluation, and perturbation-based evaluation.Source: §2 · p. 3

Consistency of LMs

• Consistency traditionally denotes invariant model behavior under meaning-preserving input alternations, whereas PertEval uses it as a technical measurement of real knowledge capacity.Prior work evaluates or intervenes on consistency as the research subject.Source: §2 · p. 3

Adversarial Text Attacks on LMs

• Adversarial text attacks seek wrong or toxic outputs from small perturbations, but PertEval introduces knowledge-invariant perturbations for trustworthy knowledge-capacity evaluation.The paper positions this as a supplement to text-attack research rather than the same objective.Source: §2 · p. 3

3.1 Knowledge-invariant Perturbations

• Content-level knowledge-invariant paraphrasing rewrites each question sentence by sentence while preserving prerequisite context and underlying knowledge.The goal is to change phrasing substantially to mitigate contamination without changing the tested knowledge.Source: §3.1 · p. 4
• Format-level perturbations test robustness under varied conditions by reordering options, changing option formatting or IDs, changing question type, and swapping question position.These strategies target option-order, option-format, selection, question-type, and global-order biases.Source: §3.1 · p. 4

3.2 Knowledge Invariance Verification

• Knowledge invariance is scored by human volunteers and superior LLM referees using predefined standards applied to original–perturbed question pairs.The final score is the average of referee outputs.Source: §3.2 · pp. 4–5, Table 1 · p. 5
• A second verification approach compares LLM performance on original and perturbed versions of questions that all evaluated LLMs answer correctly as a proxy for mastered questions.This tests whether presumed mastered knowledge remains stable after perturbation.Source: §3.2 · p. 5

2. Apply a perturbation to each sample in Dsimple to create the perturbed dataset D′

• The question-answering consistency test assesses whether a perturbation is knowledge-invariant by comparing a knowledgeable LLM’s performance on raw and perturbed datasets.No significant performance difference supports treating the perturbation as knowledge-invariant.Source: §2 · p. 5

3.3 Response Consistency Analyses for Measuring Real Knowledge Capacity

• PertEval’s response consistency analyses combine calibrated knowledge-capacity measurement with fine-grained diagnoses of vulnerability to perturbation strategies.The analyses compare performance on raw versus perturbed test sets.Source: §3.3 · p. 5
• Consistent Accuracy evaluates whether an LLM answers both the raw question and its most complex knowledge-invariant perturbation correctly.The metric treats stable correctness across question versions as evidence that the underlying knowledge has been mastered.Source: §3.3 · pp. 5–6
• Performance Drop Rate measures the accuracy discrepancy between original and perturbed datasets, with PDR < 0 indicating reduced performance after perturbation.The authors additionally apply a Wilcoxon signed-rank test to paired original and perturbed questions.Source: §3.3 · p. 6
• Recall of Performance measures the proportion of initially correct responses that remain correct after perturbation.ROP ranges from 0 to 1 and supports comparisons between LLMs, but has no absolute threshold for judging goodness.Source: §3.3 · p. 6

4.2 Real Knowledge Capacity Evaluation

• On perturbed MMLU data, gpt-4-turbo, gpt-3.5-turbo, and gemini-1.0-pro each decline by more than 25%, while gpt-4-turbo achieves only 0.555 ACC@Consist, indicating static benchmarks overvalue knowledge capacity.Evaluation uses C-Math, W-History, P-Psychology, and P-Medicine, with six representative open- and closed-source LLMs.Source: §4 · pp. 6–7, §4.2 · p. 8
• ACC@Consist exceeds the 0.0625 random-guessing baseline for every tested LLM, but most models remain below 0.5, indicating unevenly mastered knowledge.The random-guessing expectation assumes four options and one correct answer.Source: §4.2 · p. 8

4.3 Response Pattern Analysis

• Performance declines mainly because gpt-4-turbo selects extra incorrect options more often on perturbed questions, increasing from 12% to 21.7%.The analysis links this pattern to difficulty distinguishing uncertain knowledge and filtering distracting options, alongside possible rote memorization on original questions.Source: §4.3 · pp. 8–9

4.4 Overall Performance Stability

• All LLMs show negative macro-PDR for knowledge-invariant paraphrasing, and perturbations usually reduce performance by 0.04 to 0.1 on average.SwapPos is especially influential, plausibly because global order changes disrupt autoregressive generation despite preserving knowledge-relevant content.Source: §4.4 · p. 9, Table 5 · p. 9

4.5 Correct Response Consistency

• gpt-4-turbo has the best correct-response consistency, with average macro-ROP of 0.9, whereas other LLMs are vulnerable to nearly all perturbation strategies.SwapPos again has the strongest effect, exposing sensitivity to global order perturbations.Source: §4.5 · pp. 9–10, Table 6 · p. 10

5 Discussion

• PertEval exposes substantial overestimation and uncertainty in static-benchmark knowledge evaluations, while response consistency reveals vulnerabilities to knowledge-invariant perturbations.The authors also report preliminary evidence that PertEval-generated data can improve robustness through supervised fine-tuning.Source: §5 · p. 10
• PertEval is limited to close-ended benchmarks, lacks precise control over paraphrase characteristics, and has conclusions based on a subset of MMLU.Because the toolkit is open-sourced, adaptive optimization remains a risk, although perturbation diversity and changing strategies may reduce its influence.Source: §5 · p. 10

A Related Work

• LLM evaluation must address both test-data contamination and response-generation biases to measure knowledge capacity reliably.Existing work has studied contamination prevention and prompt-feature biases, but avoiding these biases while reusing valuable existing test knowledge remains underexplored.Source: A Related Work · p. 17

A.1 Benchmarks for Knowledge Capacity Evaluation

• Knowledge-capacity benchmarks use expert-designed datasets, especially multiple-choice questions, to quantify LLMs’ ability to master and apply professional knowledge.This evaluation targets solving real-world problems across general and specialized domains.Source: §A.1 · p. 17
• General benchmarks cover diverse taxonomies and domains, including MMLU’s 57 subjects, Chinese knowledge in C-Eval, human-centric exams in AGIEval, and knowledge styles in ARC.These benchmarks organize labelled datasets around particular aspects of knowledge capacity.Source: §A.1 · p. 17
• Professional benchmarks assess clinical, social-science, tax-law, scientific, mathematical, and educational knowledge through expert-designed test data.Examples include MedMCQA, PubMedQA, computational social-science tasks, tax-law questions, JEEBench, GSM-8K, and ScienceQA.Source: §A.1 · p. 17

A.2 Trustworthiness of Knowledge Capacity Evaluation

• Evaluation trustworthiness is threatened by test-data contamination, which can overvalue knowledge capacity, and by response biases toward prompt features such as option ordering.Prior work mainly prevents contamination or detects response biases, leaving reliable reuse of existing knowledge information as an open question.Source: §A.2 · pp. 17–18

B.1 Content-level Perturbation: Knowledge-invariant Paraphrasing

• Knowledge-invariant paraphrasing rewrites question sentences sequentially with an LLM rewriter while preserving the question’s knowledge content.The rewriter receives prior context, appends each perturbed sentence, and uses an expected similarity score fixed at 0.6 in experiments.Source: §B.1 · p. 18

B.2 Format-level Perturbation: Question Format Refactoring

• Format-level perturbations refactor question presentation while requiring semantic, reasoning, and answer invariance plus statement clarity.An LLM-based grader evaluates these requirements and assigns a total knowledge-invariance score using specified criteria.Source: §B.2 · pp. 18–21, Table 8 · p. 19, Table 9 · p. 20

B.3 Correct Response Consistency

• Response consistency is organized with a transition matrix that records whether answers remain consistent and correct across evaluation conditions.The matrix uses C/I for consistent/inconsistent responses and C/W for correct/wrong responses.Source: §B.3 · p. 21, Table 10 · p. 21

C.1 Details of Dataset

• The methodology reports dataset statistics for the evaluation datasets used in the study.These statistics are presented in Table 11.Source: §C.1 · p. 21, Table 11 · p. 21

C.2 Evaluation Setting

• Close-sourced models are evaluated through APIs at temperature 0.2, while open-sourced models are locally deployed with default settings.The evaluated models include gpt-4-turbo, gpt-3.5-turbo, glm-3-turbo, gemini-1.0-pro, llama-3-8b-instruct, and mistral-7b-instruct-v0.2.Source: §C.2 · p. 22
• Prompt templates generate multiple-choice and multiple-judgement questions, after which outputs are converted to JSON and regular expressions improve parsing robustness.The corresponding prompt examples are shown in Figures 8 and 9.Source: §C.2 · p. 22, Figure 8 · p. 22, Figure 9 · p. 22
• PertEval-generated data is used to fine-tune llama-3-8b-instruct with LoRA for three epochs on 479 examples.The training uses batch size 1, a learning rate of 5e-5, and a single GPU.Source: §C.2 · p. 22
• Figure 10 reports LLM accuracy on original datasets and includes gpt-4-turbo’s accuracy on the mastered question set for checking knowledge invariance.The mastered-set accuracy is defined in Section 3.2.Source: Figure 10 · p. 23

• Four experiment stages require token costs: benchmark generation, knowledge-invariance validation, the main ACC@Consist experiment, and fine-grained perturbation analysis.Source: §C.3 · p. 23
• Token costs are reported in Table 12, while USD costs per experiment are reported in Table 13.Source: §C.3 · p. 23, Table 12 · p. 23, Table 13 · p. 23
• Proprietary-model costs depend on API input/output tokens, whereas open-source-model costs depend on server runtime.Source: §C.3 · p. 23

D Verifying the Knowledge Invariance of Perturbation Strategies

• Proposed perturbations usually achieve knowledge-invariance scores above 4.5 and exceed the baseline, while KnInvPara has near-zero PDR despite lower scores.KnInvPara’s lower score reflects the unavoidable semantic changes associated with paraphrasing questions.Source: D Verifying the Knowledge Invariance… · p. 24
• Levenshtein distance shows no obvious correlation with knowledge-invariance scores, indicating that knowledge-relevant and string-level features can be partly disentangled.PromptAttack can have low edit distance yet relatively low knowledge invariance, while showing the highest distance and lowest score in W-History.Source: D Verifying the Knowledge Invariance… · p. 24
• SwapPos can reduce GPT-4-turbo performance even on mastered questions, with PDR strongly correlated with the number of tokens per question.Source: D Verifying the Knowledge Invariance… · p. 24

D.1 Probing LLMs’ Consistent Knowledge Capacity

• For k-option multiple-choice questions, pure guessing has expected ACC@Consist of 1/k^2, providing the baseline for probing consistently mastered knowledge.The proof models correct selections on both original and perturbed questions as independent events with probability 1/k^2.Source: §D.1 · p. 24

D.2 Response Pattern Analysis for LLMs in PertEval

• Response-pattern analysis compares original and perturbed data using single-correct, single-wrong, invalid, and multiple-selection categories, including whether the correct option is selected.The ratios are reported for the selected LLMs across all datasets in Figures 4 and 11–15.Source: §D.2 · pp. 24–25, Figure 11 · p. 25, Figure 12 · p. 25

D.3 How to Enhance LLMs’ Knowledge Capacity Using PertEval?

• For format-level perturbations, fine-tuning on a small subset of perturbed data can improve performance stability across perturbed data and knowledge domains.The authors attribute this stimulation phenomenon to LLMs acquiring format-level perturbations.Source: §D.3 · p. 26
• For content-level perturbations, the authors recommend domain-specific knowledge expert models trained on both original and perturbed data because transfer across datasets is limited.This recommendation balances domain variation with training cost and efficiency.Source: §D.3 · p. 26

Checklist

• Because repeating experiments across many settings was too costly and time-consuming, the study used τ = 0.2 to reduce LLM-output uncertainty and open-sourced its code for reproducibility.Assuming 6 models, 6 perturbations, 4 datasets, and 10 runs, this represented at least 1,440 experiment groups.Source: Checklist · p. 27