PaperlayerView exact revision on arXiv

Source-linked AI summary

Judging by the Cover: Cleaning LLM Truthfulness Benchmarks to Avoid Surface-Level Feature Leakage

Foad Namjoo, Remy Ogasawara, Amirali Abdullah, Cullen Anderson, Narmeen Fatimah Oozeer, Jeff M. Phillips

arXiv:2609.13003v1cs.CLcs.LG

TL;DR

Binary-choice truth benchmarks can reward systematic answer-form differences rather than intended reasoning, including in widely used TruthfulQA. The paper audits this leakage with surface features and applies Audit-Prune to reduce it in a released TruthfulQA subset.

  • Problem

    Binary-choice evaluations can let systematic surface differences between correct and incorrect answers yield above-chance accuracy without intended reasoning.

    Source: §1 · pp. 1–2

  • Method

    The paper audits TruthfulQA and other benchmarks with six surface-level answer features and uses Audit-Prune to remove pairs that most reinforce those features.

    Source: §1 · p. 2

  • Results

    Several of 14 benchmarks show substantial leakage, and HaluEval QA and MedHallu exceed TruthfulQA; the six-feature audit’s AUC on TruthfulQA-476 is 0.528 (p = 0.048).

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    TruthfulQA-476 is released as a truthfulness evaluation subset, while SURFACE6 and Audit-Prune are intended as pre-release diagnostic and repair tools for paired binary-choice datasets.

    Source: §1 · p. 3, §4 · p. 11

    The audit uses a narrow, hand-designed answer-only feature family; low audit AUC does not certify a subset as free of leakage.

    Source: §4 · p. 12

Abstract

from arXiv · show

Binary-choice truth benchmarks ask models to choose between a correct and an incorrect answer, but if the two answers differ systematically in surface-level features, models can exceed chance without performing the intended reasoning. We show that this failure mode is detectable and can be exploited by downstream classifiers. In TruthfulQA, a simple six-feature logistic classifier achieves substantial accuracy in separating correct from incorrect answers. We further show that similar surface-level artifacts are present in additional benchmarks. To counteract this, we developed a general mechanism to clean them by removing the most leakage-reinforcing pairs. We release a version of TruthfulQA with surface-feature leakage reduced close to chance and provide a mechanism, Audit-Prune, so that the datasets can be cleaned before release.

1 Introduction

Binary-choice benchmarks can reward answer-form shortcuts rather than intended reasoning, making surface leakage consequential in widely used evaluations such as TruthfulQA. The paper audits these shortcuts with interpretable features and introduces Audit-Prune to reduce them in TruthfulQA and other benchmarks.

1 Introduction

  • TruthfulQA is widely used in testing pipelines, evaluation suites, and public leaderboards, so answer-form shortcuts could distort prominent model comparisons.The binary-choice redesign addresses multiple-choice shortcuts but does not guarantee paired answers are balanced in format.Source: §1 · pp. 1–2
  • With six surface features, logistic regression separates TruthfulQA’s correct and incorrect answers at grouped-CV accuracy 0.689; several of 14 benchmarks show substantial leakage, with HaluEval QA and MedHallu exceeding TruthfulQA.The features are applied to answer strings.Source: §1 · p. 2
  • Audit-Prune removes pairs that most reinforce surface features, producing a dataset with controlled leakage; Figure 1 depicts iterative scoring, removal, refitting, and add-back.The mechanism is also proposed for auditing and cleaning other datasets.Source: §1 · p. 2, Figure 1 · p. 2
  • The six-feature audit’s AUC on TruthfulQA-476 is 0.528 (p = 0.048); the 476-pair subset retains truthfulness assessment ability while being less fooled under adversarial settings.The paper reports that the audit barely separates its answers.Source: §1 · p. 3

1.1 Related Work

  • Prior NLP work documented annotation artifacts [Gururangan et al., 2018]; this paper focuses on paired-choice auditing with a fixed, interpretable surface-feature family and pruning.The authors also describe the features as applicable to other evaluation families and usable to generate more balanced subsets.Source: §1.1 · pp. 3–4
  • Unlike AFLite’s opaque embeddings and predictability minimization, Audit-Prune targets a specified leakage threshold while preserving model rankings; pruning retains original verified pairs rather than rewriting them.The cited comparison describes rewriting as requiring truth labels to be re-verified.Source: §1.1 · p. 3

2 Detecting Surface-Level Leakage

This section audits whether answer-surface traits predict correctness, using SURFACE6 on TruthfulQA and other benchmarks. The probe identifies leakage in TruthfulQA and varying levels of leakage across other datasets.

2.1 Auditing TruthfulQA

  • 0.689 accuracy and 0.715 AUC: a question-paired-fold logistic regression using SURFACE6 separates correct from incorrect TruthfulQA answers.SURFACE6 uses surface traits involving negation, hedging, and length, without embeddings or pretrained models.Source: §2 · p. 4, §2.1 · p. 5
  • Negation count has the largest feature-level effect; removing the negation group lowers AUC from 0.715 to 0.592.The authors caution that TruthfulQA prompts concern common misconceptions, where negative answers may be semantically appropriate.Source: §2.1 · p. 5

2.2 Cross-Dataset Audit

  • Across 15 datasets, SURFACE6 results range from HaluEval QA at AUC 0.973 to BoolQ at 0.525 and PIQA at 0.509.Figure 3 summarizes the cross-dataset audit, including the released TruthfulQA-476 subset.Source: §2.2 · p. 5, Figure 3 · p. 6
  • HaluEval QA’s AUC is 0.973 versus TruthfulQA’s 0.715; hallucinated answers average 11.2 words versus 2.3 for accurate answers.On HaluBench’s 10,000-example verbatim HaluEval QA slice, a probe that reads neither questions nor retrieved context reaches AUC 0.971.Source: §2.2 · p. 6
  • MedHallu reaches AUC 0.821, while MultiNLI reaches 0.641; SNLI, MultiRC, SelfCheckGPT, and FEVER 1.0 each reach at least 0.578.For SelfCheckGPT and OpenBookQA, high accuracy can match the majority-class baseline, so the authors treat AUC as the primary leakage measure.Source: §2.2 · p. 6
  • SNLI reaches AUC 0.591, and these observations indicate that models could score above chance without knowledge of the underlying content.ANLI, VitaminC, BoolQ, and PIQA show decreasing and lesser leakage from SURFACE6 features.Source: §2.2 · p. 6

3 Cleaning Binary-Choice Benchmarks

Audit-Prune removes pairs that reinforce detectable surface-feature asymmetries, seeking the largest retained subset below a chosen audit-AUC threshold. The authors use it to produce TruthfulQA-476 and test its ranking fidelity, leakage reduction, and performance under surface inversion.

3 Cleaning Binary-Choice Benchmarks

  • Audit-Prune seeks the largest subset whose grouped-CV audit AUC is at or below threshold θ, iteratively removing the most leakage-reinforcing pairs and then adding back eligible pairs.Pair scores weight within-pair feature gaps by the classifier’s feature weights; the method refits the audit after removals.Source: §3 · pp. 7–8

3.1 Constructing TruthfulQA-476 and Ablation

  • At θ=0.53, TruthfulQA-476 retains 476 pairs with audit AUC 0.528 (95% CI [0.503, 0.557], p=0.048), after broad pruning across both question types.All 37 categories retain at least one pair.Source: §3.1 · p. 9

3.2 Evaluation-Fidelity Check

  • TruthfulQA-476 correlates with full-benchmark model rankings at Spearman ρ=0.915, within the variation observed across random 476-pair subsets.Across audit-pruned subsets, Kendall’s τ is at least 0.81 and Spearman’s ρ exceeds 0.91.Source: §3.2 · p. 9

3.3 Comparison to Adversarial Filtering (AFLite)

  • At N=476, Audit-Prune leaves surface-audit AUC 0.528 versus AFLite’s 0.603, with comparable ranking fidelity (ρ ∈[0.91, 0.98]).AFLite removes pairs predictable from frozen embeddings; Audit-Prune targets six interpretable surface cues.Source: §3.3 · p. 9, Table 2 · p. 10

3.4 Surface-Inversion Generalization Test

  • On surface-flipped adversarial prompts, several classifiers trained on full TruthfulQA score below 0.5, while cleaned training raises their truthfulness accuracy; SURFACE6 itself scores 0.The evaluation uses 131-prompt Adversarial and Natural cohorts; adversarial responses invert surface cues, while natural responses are not shaped for them.Source: §3.4 · pp. 9–11, Table 3 · p. 10

3.5 IRT-Selected Anchors Remain Surface-Leaky

  • The IRT-selected anchors retain surface leakage at about the same rate as a 100-pair random subset, so IRT selection does not resolve this confound.Four time-sensitive items and one duplicate leave 95 relevant evaluation pairs.Source: §3.5 · p. 11, Table 4 · p. 11

4 Artifact, Intended Use, Limitations, and Discussion

The authors release TruthfulQA-476, a cleaned 476-pair subset, and describe its intended use alongside limitations of the audit and cleaning approach. They propose integrating surface audits into dataset creation while anticipating further feature variants.

Artifact

  • Surface-form audit accuracy falls from 0.689 to 0.522 on TruthfulQA-476, with model-ranking agreement of Spearman ρ = 0.915 and Kendall τ = 0.827 against the full benchmark.The release includes the cleaned CSV, pair_id manifests, thresholded subsets, fixed-prefix baselines, two evaluation cohorts, and code.Source: §4 · p. 11

Intended Use

  • TruthfulQA-476 is intended to replace the binary TruthfulQA evaluation set, while SURFACE6 and Audit-Prune are intended as pre-release diagnostics and repair tools for paired-choice datasets.Alternative feature families may be designed for specific tasks; whether SURFACE6 or other features fit other datasets remains future work.Source: §4 · p. 11

Limitations

  • The answer-only SURFACE6 audit is narrow: low audit AUC bounds leakage expressed by that feature family but does not certify a subset as leakage-free.The features are hand-designed and TruthfulQA-motivated, and the released lexicons are English-specific.Source: §4 · p. 12
  • Audit-Prune is greedy rather than globally optimal, and some cues it removes, especially negation, may be semantically appropriate for truthfulness tasks.The paper describes tractable globally optimal subset selection under a non-linear AUC constraint as an open combinatorial problem.Source: §4 · p. 12
  • The released TruthfulQA-476 contains only pairs from the public source benchmark and inherits its memorization and contamination exposure.Source: §4 · p. 12

Discussion

  • The paper expects larger surface-feature sets to detect more leakage but notes that they may let Audit-Prune overfit.It argues for integrating automated surface audits into dataset creation to catch patterns in both hand-written and LLM-generated data before release.Source: §4 · p. 12

Reproducibility statement

The reproducibility materials include released data and scripts, a specified audit protocol, and records of evaluation configurations and outputs.

Reproducibility statement

  • The repository provides the TruthfulQA-476 manifest and CSV, both auxiliary cohorts with their verification sheet, per-item model predictions, and a script for each table and figure.Source: Reproducibility statement · p. 13
  • The audit protocol specifies grouped five-fold cross-validation, StandardScaler with ℓ2 logistic regression, permutation nulls, and cluster-bootstrap intervals.The primary audits use B=10,000 permutations, with smaller values where stated.Source: Reproducibility statement · p. 13
  • The repository pins package versions and records random seeds, while closed-model evaluations include exact API model identifiers, evaluation dates, and cached responses.Source: Reproducibility statement · p. 13

Ethics statement

The authors describe the source and licensing of the released materials, verification steps for auxiliary cohorts, and the audit’s potential dual use.

Ethics statement

  • TruthfulQA-476 contains original TruthfulQA pairs and is released under the source dataset’s Apache-2.0 license.Source: Ethics statement · p. 13
  • The two auxiliary cohorts are LLM-generated from prompts, labeled as synthetic, and verified against written sources, with a verification sheet released to trace residual labeling errors.Source: Ethics statement · p. 13
  • The surface-cue probe is released for auditing, although the authors note it could in principle be used to construct adversarial items; no personal data is involved.Source: Ethics statement · p. 13

AI-use statement

The authors disclose LLM assistance across synthetic-data generation, research critique, implementation, and writing, with human verification or review described for each use.

AI-use statement

  • LLM generator and judge models generated and pre-screened the two auxiliary cohorts, and the authors subsequently verified every released pair against sources.Source: AI-use statement · p. 13
  • LLM systems critiqued experimental design, related-work coverage, and interpretation, and the authors independently verified suggestions before adoption.Source: AI-use statement · p. 13
  • LLM tools assisted with analysis scripts, figures, literature search, and readability editing, with all code reviewed and executed by the authors.Source: AI-use statement · p. 13

A Additional Cleaning Results

The appendix compares alternative Audit-Prune scoring strategies and describes an AFLite comparison protocol.

A Additional Cleaning Results

  • At every tested threshold, imbalance retains more pairs than confidence, so the authors use imbalance as the main method.The confidence score is the audit classifier’s difference in grouped out-of-fold predicted probabilities for each pair’s two answers.Source: A Additional Cleaning Results · p. 18
  • At θ=0.55, the α=0.5 hybrid retains 559 pairs versus 492 for pure imbalance, but at θ=0.53 it retains 386 versus 476.The gains are not uniform, suggesting the greedy imbalance construction is not globally optimal.Source: Table 6 · p. 18, A Additional Cleaning Results · p. 18

B AFLite Comparison Details

  • AFLite scores pairs using mean held-out predictability from 64 logistic classifiers trained on random pair-disjoint half/half partitions.It removes the 10 most predictable pairs per round down to 300 pairs; predictability is accuracy-based, not AUC-based.Source: B AFLite Comparison Details · p. 19

C TruthfulQA Feature-Group Ablations

The feature-group ablations identify negation as the strongest SURFACE6 signal and test whether other features or a broader negation lexicon add information.

C TruthfulQA Feature-Group Ablations

  • Dropping avg_token_len and type_token lowers AUC from 0.715 to 0.691 (∆= −0.023), about five times the 0.005 rename threshold.The authors therefore retain SURFACE6 rather than renaming the feature set SURFACE4.Source: C TruthfulQA Feature-Group Ablations · p. 19, Table 7 · p. 19
  • Extending both negation lexicons with 13 synonym-negation tokens changes grouped-CV AUC by less than 0.0002.The authors note that TruthfulQA reference answers express negation mostly through basic forms such as no, not, never, and contractions.Source: C TruthfulQA Feature-Group Ablations · p. 19, Table 7 · p. 19
  • Removing negation features lowers AUC from 0.715 to 0.592, the largest drop among the feature-group ablations.The authors interpret this as evidence that negation carries most of the surface-form signal available to the audit classifier.Source: Table 7 · p. 19, C TruthfulQA Feature-Group Ablations · p. 19
  • Table 8 reports per-token leave-one-out ablations of the canonical NEG_WORDS lexicon under SURFACE6.Each row removes one token and, where applicable, its single-word lead pattern; rows are sorted by absolute AUC deviation from the full model.Source: Table 8 · p. 20, C TruthfulQA Feature-Group Ablations · p. 20

D Classifier and Cohort Construction Details

The surface-inversion test uses conventional pooling choices for encoder families and attention-masked mean pooling for six causal-LM families.

D Classifier and Cohort Construction Details

D.1 Surface-flipped cohort (n=131)

The surface-flipped cohort contains 131 TRUE/FALSE pairs designed to reverse TruthfulQA’s usual surface cues, constructed through a revised generation prompt and verification pipeline.

D.1 Surface-flipped cohort (n=131)

  • The 131 pairs span twelve misconception categories, with FALSE answers carrying cues associated with TRUE TruthfulQA answers and TRUE answers written as bare positive assertions.The FALSE side has sentence-initial negation, hedging, and authority cues; the categories overlap with but are not identical to TruthfulQA’s.Source: §D.1 · p. 20, Table 9 · p. 21
  • Eleven batches were generated with claude-opus-4-5 as the primary model; the configured fallback models were not used.The generation prompts specified the inverted surface profile and requested coverage of at least eight categories.Source: §D.1 · p. 20
  • After a 20-pair pilot exposed cases where negation made the FALSE answer factually true, the authors revised the prompt to preserve the misconception proposition.The remaining ten batches used the revised template.Source: §D.1 · p. 20
  • Pairs passed surface and polarity checks, an independent truth judge, and question de-duplication before source verification.The drift screen was advisory: 40 of 111 post-pilot pairs carried a drift flag and were retained, with author source verification serving as the final label-correctness gate.Source: §D.1 · p. 21, Table 9 · p. 21

D.2 TRUTHFULQA-NATURAL cohort (n=131)

This appendix describes the generation and screening of TRUTHFULQA-NATURAL, a 131-pair cohort of independently generated, open-ended factual questions across 12 neutral domains, avoiding TruthfulQA-specific inputs and yes/no formats. It reports that the author-verified pairs were selected from 179 judge-admitted candidates to match the surface-flipped cohort’s per-topic counts, and gives examples of rewrites that passed or failed drift review.

D.2 TRUTHFULQA-NATURAL cohort (n=131)

  • The 131-pair cohort was finalized from 135 selected pairs after four removals; its questions span 12 neutral domains unrelated to TruthfulQA’s categories.Of 182 candidates, 179 passed joint TRUE/FALSE judging at confidence ≥0.8; the first N per topic were selected to match the surface-flipped cohort’s per-topic counts before author verification removed four pairs.Source: §D.2 · pp. 21–22
  • The generator was not shown TruthfulQA, the surface-flipped cohort, or its feature lexicons and targets; yes/no probes were excluded to avoid preloading a negation-on-truth-side signal.The answer-pair prompt instructed the model not to consider negation, hedging, length, or authority phrasing.Source: §D.2 · pp. 21–22

E Cohort Generation Prompts

  • The reproduced cohort-generation prompt references the earlier Surface10 profile, while the paper’s audits use only six SURFACE6 features.The canonical post-pilot template generated 115 of 135 selected pairs and 111 of the released 131; the earlier template generated the remaining 20 selected pairs.Source: E Cohort Generation Prompts · p. 22, §E.1 · p. 22

E.1 Prompt 1 — Surface-flipped cohort: pair generation

  • The surface-flipped prompt’s proposition-preservation rule required FALSE answers to assert the misconception, using negation only as surface form rather than turning the answer into a factual debunking.The prompt identifies denial of the misconception as a failure because it flips the false answer into a true one.Source: §E.1 · p. 25

E.2 Prompt 2 — Surface-flipped cohort: LLM judge

  • The natural cohort reused the surface-flipped cohort’s judge prompt with the same model and parameters; the confidence-≥0.8 admission rule was applied during cohort finalization.The judge prompt was issued once per candidate pair surviving upstream mechanical and polarity screens.Source: §D.2 · p. 22, §E.2 · p. 26
  • The surface-flipped judge admitted pairs only when it confirmed both intended truth values, the same question, and confidence of at least 0.8.The TRUE side carried the typical FALSE-side surface profile, while the FALSE side carried the typical TRUE-side profile.Source: §E.2 · p. 26

E.3 Prompt 3 — TRUTHFULQA-NATURAL cohort: question generation

  • 182 candidate questions were generated across 12 topic domains; the starter filter and deduplication yielded 182 accepted candidates, one fewer than requested.The shortfall was in Modern history and society, with 7 of 8 requested questions accepted; all candidates came from claude-opus-4-5.Source: §E.3 · p. 27

E.4 Prompt 4 — TRUTHFULQA-NATURAL cohort: answer-pair generation

  • The answer-pair prompt generated natural correct and plausible-but-wrong answers together, instructed the model not to consider surface features, and sent all 182 successful pairs to judging.This constraint preserved the natural cohort as a null test of whether cleaning hurts when surface leakage is absent.Source: §E.4 · p. 27

E.5 Natural cohort: LLM judge

  • The natural cohort used the surface-flipped cohort’s judge prompt and admission rule, binding the generated natural TRUE and FALSE answers to the corresponding sides.Judging used gpt-5.4-2026-03-05 through the OpenAI Responses API with temperature 0.0 and JSON-object mode.Source: §E.5 · pp. 27–28
  • 179 of 182 natural answer pairs passed joint judge admission, and 131 author-verified pairs were released to match the surface-flipped cohort’s per-topic counts.The judge required both intended truth labels, the same question, and confidence of at least 0.8.Source: §E.5 · p. 28

E.6 Surface-flipped cohort: inspectable examples

  • Inspectable surface-flipped examples include an admitted rewrite and a drift-flagged rewrite; the latter changed the claim’s subject and certainty or made a stronger claim.Every released pair was source-verified by the authors, while independent third-party labels were not collected at scale.Source: §E.6 · p. 28

F Cross-Dataset Feature-Group Ablations

Removing length features reduces audit AUC across BoolQ, HaluEval QA, and VitaminC; the largest drop is on HaluEval.

F Cross-Dataset Feature-Group Ablations

  • AUC drops from 0.973 to 0.544 on HaluEval, 0.525 to 0.500 on BoolQ, and 0.550 to 0.503 on VitaminC when length features are removed.The BoolQ residual signal is carried by length+regularity; the VitaminC residual signal is weakly length-driven.Source: F Cross-Dataset Feature-Group Ablations · p. 29, Table 11 · p. 29, Table 10 · p. 29
  • The ablations use question text on BoolQ (N=3,270), answer text on HaluEval QA (N=20,000), and claim text on VitaminC (N=54,012).They use stratified CV for BoolQ and grouped CV for HaluEval QA and VitaminC.Source: Table 10 · p. 29, Table 11 · p. 29, Table 12 · p. 29
  • The cross-dataset audit table reports audited-text counts and cross-validated accuracy under each dataset’s split protocol, with permutation nulls and cluster-bootstrap confidence intervals.Source: Table 13 · p. 30
  • The threshold sweep compares Audit-Prune with fixed-prefix pruning by retained pairs, audit accuracy and AUC, and model-ranking fidelity against TruthfulQA-790.Source: Table 14 · p. 31
Loading 2609.13003v1…