PaperlayerView exact revision on arXiv

Source-linked AI summary

PROOF: Profiling Reliability of Object-Level Facts in Large Language Models

Andrei Chetvergov, Mikhail Solovev, Timofei Sivoraksha, Stepan Ukolov, Valeriia Kuschenko, Alexander Evseev, Sergey Bolovtsov

arXiv:2609.29504v1cs.CLcs.AI

TL;DR

Aggregate factuality scores can obscure where models succeed and how their answers change under controlled variations. PROOF profiles Wikidata-grounded object-level factual coverage across deployments and interventions; results show broad domain variation alongside differences in accuracy and response stability.

  • Problem

    Aggregate factuality scores can rank systems but poorly describe distinct gaps in retrieved facts and responses to changes in direction, wording, or decoding.

    Source: §1 · p. 1

  • Method

    PROOF builds a Wikidata-grounded benchmark and profiles object-level factual coverage across domains and controlled prompt and decoder interventions.

    Source: §1 · p. 2, §3.1 · p. 3

  • Results

    Across deployments, primary accuracy ranges from 6.58% to 57.59%, while every deployment's best–worst domain range is 19.29–36.39 percentage points.

    Source: S4 Exact Model-Level Summary · p. 12

  • Takeaways & Limitations

    PROOF supports profile-based interpretation: strong average accuracy need not mean uniform coverage, and answer breakage is distinct from false-hint following.

    Source: §5 · p. 7, §7 · p. 8

    PROOF tests English multiple-choice selection over Wikidata-derived facts, not open-ended precision, reasoning chains, multilingual knowledge, or all facts worth knowing.

    Source: §6 · p. 8

Abstract

from arXiv · show

Aggregate factuality scores hide where a language model succeeds, which relations it confuses, and whether an answer survives innocuous changes to the question or decoder. We introduce PROOF, a profile-oriented benchmark for factual coverage in instruction-tuned language models. PROOF converts a frozen Wikidata snapshot into 18,486 English multiple-choice questions grounded in 11,779 semantic facts, 101 classes, 392 properties, and 14 domains. Each question has an explicit "I don't know" option, a "No correct option" control, and nine controlled formulations; 1,849 questions are no-correct-option traps. We evaluate 18 open-weight model deployments on 166,374 prompts each and separately perturb decoding on a fixed 10% subset. Base factual accuracy ranges from 6.58% to 57.59% (chance: 8.64%), yet every model has a 19.3-36.4 percentage-point spread across domains. Paired facts reveal direction-dependent retrieval, usually favoring subject-to-object queries, with the pattern reversing for one model. We find no consistent temporal penalty after exact-stratum adjustment. Neutral wording changes accuracy by as much as 26.5 percentage points, while adversarial formulations break up to 79.4% of answers that were initially correct. Direct switching to an injected false label varies from 0.04% to 27.5%, showing that accuracy loss and hint following are distinct. Selected-token confidence often indicates severe overconfidence, and decoder perturbations move accuracy by up to 15.7 percentage points and domain profiles by 16.8 points. PROOF therefore measures factual coverage as a structured, intervention-aware profile rather than a single claim about what a model "believes."

1 Introduction

Aggregate factuality scores can rank systems but poorly describe what facts a deployment retrieves or how answers respond to changes in direction, wording, or decoding. PROOF profiles object-level factual coverage through controlled interventions, distinguishing behaviors that a single score can conflate.

1 Introduction

  • PROOF profiles factual coverage across domains, classes, properties, and cross-cutting categories, using paired and controlled interventions to examine distinct response behaviors.The unit is a deployment under a recorded prompt and decoder configuration; coverage is behavioral evidence, not evidence about worldview, belief, or the cause of a later opinion.Source: §1 · p. 2

1.1 Evaluation Questions

  • Five research questions examine coverage profiles, matched temporal facts, retrieval direction, wording and false-hint effects, and decoder drift.Traps and selected-token confidence provide a cross-cutting analysis of epistemic boundaries rather than another claim about factual recall.Source: §1.1 · p. 2

2 Related Work

  • PROOF complements factual probes and broader evaluation frameworks by testing grounded object-level fact selection with class/property structure and matched prompt interventions.Related work covers stored relations, long-tail and temporal knowledge, truthfulness, and factual precision in generated text; prompt and decoder variation are treated as distinct measurement changes.Source: §1 · pp. 1–2, §2 · p. 2
  • Selected-answer-token probability diagnoses confidence in the emitted choice, not a normalized forecast over valid options or a general measure of knowing what one knows.PROOF reports the probability assigned to the selected answer token.Source: §2 · p. 2

3 PROOF Benchmark

PROOF constructs a Wikidata-grounded benchmark and evaluates deployments through structured coverage profiles and controlled prompt and decoder interventions. Its design preserves semantic-fact identity, separates response behaviors, and uses paired analyses where applicable.

3.1 Scope and Dataset Construction

  • The dataset samples eligible entities and statements, records semantic-fact identifiers before expanding direction or wording, and includes DSF, RSF, descriptions, and no-correct-option traps.Traps remove the true substantive answer, making “No correct option” correct; trap performance measures invalid-option discrimination and is excluded from primary accuracy. Class–property relevance was assigned by one annotator without inter-annotator agreement measurement.Source: §3.1 · p. 3

3.2 Controlled Formulations

  • Each base question receives seven neutral and two adversarial formulations, with question text, option order, correct label, and false-hint label recorded for within-question comparisons.Adversarial conditions preserve a warning to verify the claim; the formulations alter the instruction frame, not the semantic payload.Source: §3.2 · pp. 3–4

3.3 Models and Inference

  • The study evaluates 18 instruction-tuned open-weight deployments using a fixed decoder configuration, retaining invalid outputs in denominators and reporting exact deployment identifiers.The roster is not a parameter-controlled architecture comparison; calibration accuracy is computed on the usable confidence subset and kept distinct from primary accuracy.Source: §3.3 · p. 4

3.4 Metrics and Statistical Design

  • Primary accuracy uses real base-formulation questions, while profile summaries aggregate by property, class, and domain; direction and temporal analyses use paired or exact-stratum comparisons.Chance-normalized accuracy uses item-wise random-choice accuracy; domain summaries macro-average represented classes, and temporal estimates are adjusted associations rather than causal effects.Source: §3.4 · p. 4
  • Neutral wording, adversarial breakage, strict switching to an injected false label, traps, and selected-token calibration are measured as distinct outcomes.Strict hint following requires a change specifically to the injected label; calibration reports selected-token confidence, expected calibration error, and the mean confidence–accuracy gap.Source: §3.4 · p. 4
  • Decoder robustness is exploratory: a fixed 10% subset is tested under 37 non-baseline perturbations, with accuracy drift, answer agreement, domain drift, and rank correlation reported.No stability threshold was preregistered.Source: §3.4 · p. 4

4 Results

Across deployments, aggregate accuracy coexists with substantial domain variation, while matched question and decoder changes alter measured responses. The results distinguish temporal association, directional retrieval, trap policies, calibration, and robustness rather than collapsing them into one score.

4.1 Accuracy Is Not a Coverage Profile

  • Base accuracy spans 6.58–57.59% against 8.64% chance, yet every deployment’s domain range is at least 19.3 points, reaching 36.4 for microsoft/Phi-3-medium-4k-instruct.Qwen/Qwen2.5-7B-Instruct falls below chance because it selects “I don’t know” on most items.Source: §4.1 · pp. 4–5, Table 3 · p. 5, Figure 2 · p. 5
  • Deployments separated by only 0.04 aggregate points nevertheless differ substantially in trap policies, calibration gaps, and decoder sensitivity; these are descriptive comparisons, not controlled family ablations.The deployment ranking alone misses systematic within-model structure.Source: §4.1 · p. 5

4.2 Time and Direction Separate Different Retrieval Effects

  • After exact-stratum adjustment, results show no consistent temporal penalty, although one deployment’s odds ratio barely excludes one before multiplicity correction.The adjusted analysis does not establish temporal invariance or a common recent-versus-non-recent effect.Source: §4.2 · p. 6
  • DSF-minus-RSF accuracy is usually positive but model-dependent, ranging from −6.92 points for ibm-granite/granite-3.3-8b-instruct to +7.15 for mistralai/Ministral-8B-Instruct-2410.Twelve deployments’ confidence intervals are wholly above zero, while ibm-granite/granite-3.3-8b-instruct is significantly reversed.Source: §4.2 · p. 6

4.3 Wording Can Dominate Measured Knowledge

  • Neutral wording shifts accuracy by −26.53 to +14.77 points, while adversarial formulations break up to 79.36% of answers that were correct at base.Strict switches to the injected false label range from 0.04% to 27.51% of eligible transitions, so answer breaks and hint following are distinct.Source: §4.3 · p. 6

4.4 Epistemic Boundaries and Confidence

  • Trap responses reveal distinct policies: mistralai/Ministral-3-14B selects a false substantive option on 95.73% of traps, while Qwen/Qwen2.5-7B selects “I don’t know” on 97.84%.Trap avoidance alone does not measure an epistemic boundary; transitions relative to real questions and response decomposition also matter.Source: §4.4 · p. 6
  • Selected-token confidence is often overconfident and is not normalized over valid choices; confidence quality and factual coverage are independent axes.Qwen/Qwen2.5-7B-Instruct has 6.58% accuracy and 97.92% selected confidence, with its gap partly reflecting its near-universal “I don’t know” policy.Source: §4.4 · pp. 6–7

4.5 Decoder Robustness Is Deployment-Specific

  • Across 37 non-baseline decoder conditions, rank correlations range from 0.870 to 1.000, while accuracy changes reach −15.68 points and domain-profile drift reaches 16.82 points.Seed perturbations can also change labels when aggregate drift is modest.Source: §4.5 · p. 7

5 Discussion

PROOF treats factual evaluation as a structured profile whose behavioral axes should be interpreted together. The discussion recommends reporting those axes while limiting conclusions to the benchmark and serving stack.

5 Discussion

  • PROOF separates domain-pattern diagnosis from causal explanation, distinguishes intervention types, and reads accuracy, trap behavior, and confidence jointly.The benchmark diagnoses patterns but does not identify their causes.Source: §5 · p. 7

5.1 PROOF Reporting Card

  • The minimum reporting card lists six entries; application profiles can prioritize coverage or trade it for trap discrimination, calibration, and decoder stability.Selected-token confidence belongs on the card when log probabilities are available, with its restricted interpretation stated.Source: §5.1 · p. 7, Table 4 · p. 8

5.2 Bounded Interpretation

  • Findings are conditional on the benchmark and serving stack; decoder effects are exploratory magnitudes, not stable/unstable labels, and the study does not identify mechanisms or warrant causal claims.The bounded interpretation also excludes claims about internal representations.Source: §5.2 · pp. 7–8, Figure 4 · p. 8

6 Limitations, Ethics, and Reproducibility

The paper bounds PROOF’s conclusions by its multiple-choice, Wikidata-derived design and by limits in item dependence, decoder analysis, and taxonomy judgments.

Construct validity

  • PROOF tests English multiple-choice selection over Wikidata-derived facts, not open-ended precision, reasoning chains, multilingual knowledge, or all facts worth knowing.Automated curation and type-compatible distractors can leave omissions, contested statements, and noise; multiple choice can create label priors and recognition without free recall.Source: §6 · p. 8

Statistical limits

  • Shared entities and properties can make item-wise uncertainty optimistic; decoder robustness remains exploratory, and public release creates contamination risk requiring dated revisions.The study uses class bootstrap and exact pairing where available.Source: §6 · p. 8

Scope and reproducibility

  • Taxonomy and relevance labels encode judgment, cross-cutting categories are non-additive, and the frozen artifact retains provenance details while Wikidata-derived content keeps its upstream terms.The recorded provenance includes the snapshot cutoff, hashes, taxonomy decisions, model revisions, serving settings, raw selections, and log probabilities.Source: §6 · p. 8

7 Conclusion

PROOF profiles object-level factual coverage behaviorally across 18 deployments and controlled interventions, rather than treating aggregate accuracy as a complete account of model behavior.

7 Conclusion

  • A strong average need not imply uniform coverage, a broken answer need not imply false-hint following, and stable ranks need not imply stable answers.PROOF reveals where a deployment selects object-level facts under controlled interventions, not what the model believes or why.Source: §7 · p. 8

S1 Evidence Contract and Completeness

The evidence contract documents complete main runs and decoder conditions, defines the analysis units, and records dataset and serving details alongside provenance qualifications.

S1 Evidence Contract and Completeness

  • All 18 deployment runs contain 166,374 completed prompts each, and the decoder study includes all 37 planned non-baseline conditions on a fixed subset.The configured and observed deployment sets match, and the paper’s analyses use this complete evidence state.Source: S1 Evidence Contract and Completeness · p. 10, Table S1 · p. 10
  • Primary accuracy uses real base-formulation questions, excludes traps, and macro-averages represented classes for domain accuracy; paired facts support direction and temporal analyses use exact-stratum associations.Neutral wording uses matched Cochran’s Q tests; adversarial breaks and strict false-hint following are distinct events, while decoder results are exploratory magnitudes.Source: S1 Evidence Contract and Completeness · p. 10

S2 Dataset Card

  • The frozen Wikidata snapshot is dated 20 July 2026; the final sample represents 101 classes and 392 properties from an expert-authored taxonomy.The taxonomy has 14 mutually exclusive top-level domains, 115 classes, and 13 overlapping cross-cutting categories.Source: S2 Dataset Card · p. 10
  • Question-quality scores have a median of 0.8317, and curation kept 17,770 of 20,266 candidates, rewrote 716, and rejected 1,780.The automated curation pass used Qwen/Qwen3.8-27B-FP8 at temperature zero with thinking disabled.Source: S2 Dataset Card · p. 10

S3 Exact Deployment Registry

  • Frozen main runs use vLLM 0.19.0 with bf16 precision and chat completions; output budgets differ for three deployments, while invalid outputs remain in the accuracy denominator.The maximum invalid outputs among the 16,637 primary items is 145.Source: S3 Exact Deployment Registry · p. 10

S4 Exact Model-Level Summary

The model-level summary reports accuracy, chance-normalized accuracy, domain range, trap selection, confidence gap, and decoder profile drift. Coverage varies substantially across domains, with different strongest and weakest domains across deployments.

S4 Exact Model-Level Summary

  • Table S4 defines the common axes as item-wise chance-normalized accuracy, best-minus-worst macro-domain accuracy, and maximum mean absolute drift across 14 macro-domain accuracies.The accompanying model-level table is organized by exact upstream identifier and reports accuracy, trap-selection probability, confidence gap, and maximum profile drift.Source: S4 Exact Model-Level Summary · pp. 11–12, Table S4 · p. 11

S5 RQ1: Hierarchical Factual Coverage

  • Primary accuracy ranges from 6.58% to 57.59% against 8.64% item-wise chance, and every deployment’s best–worst domain range is 19.29–36.39 percentage points.Psychology and behavior is strongest for 17 deployments; media, information and communication is weakest for 14.Source: S4 Exact Model-Level Summary · p. 12
  • The coverage atlas marks domain-specific random-choice chance, while the 18-by-14 heatmap shows macro-domain percentages for each deployment.The companion class and cross-cutting views show deployment-distinctive classes; cross-cutting categories overlap and must not be summed.Source: Figure S1 · p. 12, Figure S2 · p. 13, Figure S3 · p. 14

S6 RQ2 and RQ3: Temporal Association and Direction

Matched temporal facts show no consistent adjusted penalty, while paired facts reveal a common but non-universal directional retrieval asymmetry. Formulation changes and epistemic-boundary measures expose distinct response patterns that should be interpreted jointly.

S6 RQ2 and RQ3: Temporal Association and Direction

  • Temporal common odds ratios range from 0.806 to 1.331, with 17 of 18 confidence intervals including one after exact-stratum adjustment.The strata control for sampling class, property, direction, answer kind, option count, and quality quintile; the single interval barely excluding one is not treated as a cross-deployment finding.Source: S6 RQ2 and RQ3: Temporal Association… · p. 14
  • Paired DSF-minus-RSF accuracy ranges from −6.92 to +7.15 points across 1,272 facts, with 12 wholly positive confidence intervals and one wholly negative.The expanded figure pairs effects with joint DSF/RSF outcomes; these comparisons establish retrieval asymmetry, not its cause.Source: S6 RQ2 and RQ3: Temporal Association… · p. 14, Figure S4 · p. 15

S7 RQ4: Formulation Sensitivity and False-Hint Following

  • Neutral-formulation accuracy deltas range from −26.53 to +14.77 points, while adversarial variants break 9.18% to 79.36% of base-correct answers.Strict direct false-hint following is narrower, ranging from 0.043% to 27.508% among eligible transitions; other substantive wrong answers are not counted as direct hint following.Source: S7 RQ4: Formulation Sensitivity and… · p. 15, Figure S5 · p. 16, Figure S6 · p. 17

S8 Epistemic Boundary and Selected-Token Calibration

  • Trap-selection probability ranges from 0 to 43.37%, and selected-token calibration gaps range from 2.29 to 91.33 percentage points.Trap scores require joint interpretation with abstention and substantive-error rates because blanket abstention can reduce trap errors while depressing ordinary accuracy; confidence is the emitted label-token probability without option-set renormalization.Source: S8 Epistemic Boundary and… · p. 17, Figure S7 · p. 18

S9 RQ5: Decoder Robustness

Decoder perturbations preserve broad deployment ranks but can substantially shift individual accuracy and domain profiles. The operational profile reports prompt-token volume and median answer latency, without inferring hardware or cost.

S9 RQ5: Decoder Robustness

  • The fixed decoder subset contains 1,849 base-formulation questions and evaluates 20 seeds, eight top-k values, and a balanced nine-cell temperature/top-p/top-k grid.The reused main baseline is separate, yielding 37 non-baseline conditions and 38 rows per deployment including baseline.Source: S9 RQ5: Decoder Robustness · p. 18, Figure S8 · p. 19, Figure S9 · p. 20
  • Across decoder perturbations, accuracy drift ranges from −15.68 to +9.26 points and maximum mean absolute domain-profile drift from 0.93 to 16.82 points.Rank correlations with the main baseline range from 0.870 to 1.000 (mean 0.953), so stable aggregate ranks do not imply stable answers or domain profiles.Source: S9 RQ5: Decoder Robustness · p. 18, Figure S10 · p. 21

S10 Operational Profile

  • The operational profile reports recorded prompt-token volume and median answer latency, with the latency axis logarithmic.Hardware, energy, wall time, and monetary cost are not inferred.Source: Figure S12 · p. 23

S11 Hashes, Licensing, Ethics, and Reproduction

The appendices describe reproducibility checks and publication-figure provenance, while noting limits tied to benchmark representation, contamination risk, and preserved evidence. PROOF is recommended as a versioned regression and auditing instrument, not a universal factuality or alignment statement.

S11 Hashes, Licensing, Ethics, and Reproduction

  • Reproduction requires verifying source hashes and the 18-deployment roster, retaining invalid responses in denominators, and preserving paired identifiers and the fixed decoder subset.With those checks, results reproduce through deterministic post-processing without contacting a model endpoint.Source: S11 Hashes, Licensing, Ethics, and… · p. 23
  • The publication pipeline builds figures from frozen result archives’ machine-readable tables and run registry, verifies the identical roster, and records source hashes in a manifest.It does not read pre-rendered plot directories.Source: S11 Hashes, Licensing, Ethics, and… · p. 23
  • The taxonomy and class–property relevance labels reflect expert judgment, Wikidata may contain social, political, and historical biases, and public release creates eventual contamination risk.The authors recommend PROOF as a versioned regression and auditing instrument rather than a universal statement about factuality or alignment.Source: S11 Hashes, Licensing, Ethics, and… · p. 23

S13 Remaining Evidence Limits

  • Hardware, energy, experiment wall time, and monetary cost remain unavailable, and multiple-choice selection does not establish open-ended factual precision, internal belief, worldview, or causal grounding.Four historical deployments have selected-token confidence coverage below 100%, making calibration-subset accuracy distinct from primary accuracy; fourteen historical manifests record dirty source worktrees.Source: S13 Remaining Evidence Limits · p. 24
Loading 2609.29504v1…