PaperlayerView exact revision on arXiv

Source-linked AI summary

VERITYGATE: A Four-Gate Schema-Level Faithfulness Framework and Paired Benchmark for Grounded LLM Narrations over Structured Evidence

Sachin Gupta

arXiv:2610.00833v1cs.CLcs.LG

TL;DR

LLM explanations of structured outputs may depart from their evidence, including by citing nonexistent entities or distorting numbers. VERITYGATE checks declared claims against backend evidence through four structural gates. After one repair pass, claim-type rules account for nearly all failing claims across the tested models.

  • Problem

    LLM narrators given full structured evidence may still cite nonexistent cards, distort numbers, flip signs, or invent plausible facts.

    Source: §1 · p. 1

  • Method

    VERITYGATE applies four claim-level checks for evidence existence, entity allowlisting, number binding, and claim-type rules against structured backend evidence.

    Source: §1 · p. 1, §2 · p. 2

  • Results

    After one repair pass, Gate 4 covers 97.0%, 98.7%, and 100% of failing claims for mini, Llama, and Sonnet, respectively.

    Source: §4.2 · p. 6

  • Takeaways & Limitations

    Prose-only preference is not a substitute for mechanical faithfulness verification.

    Source: §1 · p. 2

    The three tested backends may not represent future models, so both the observed pattern and absolute numbers require replication.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Fluent LLM explanations may not follow the evidence from a structured system. We present VERITYGATE, a four-gate checker for declared evidence IDs, entities, numbers, and claim types. It checks a fixed schema; it does not verify every fact in the prose. At r=0 and r=1, we test 900 instances per setting (450 grounded-ungrounded pairs) with GPT-4o-mini, Llama-3.3-70B, and Claude Sonnet 4.6. Under this schema-level contract and before repair, 80.3% of mini claims and 47.9% of Sonnet claims fail. These are verifier rejection rates, not prose-hallucination rates. One repair pass raises claim survival from 19.7% to 28.0% for mini and from 52.1% to 54.3% for Sonnet. Verified claims per example change by +0.14 for mini, -0.71 for Llama, and -0.47 for Sonnet, so survival and output volume must be reported together. A second Sonnet pass gives no clear gain. At r=1, Gate 4 covers 97.0%, 98.7%, and 100% of failing claims for mini, Llama, and Sonnet. Small human studies support the rules but show gaps between schema checks and correct prose. A domain-specific GPT-4o judge test shows an order effect, so it is only a usefulness check. We release the code and data.

1 Introduction

LLM narrations over structured evidence can still cite nonexistent entities, distort numbers, or invent plausible facts. VERITYGATE introduces claim-level structural checks, a paired benchmark, and an inference-time repair pipeline.

1 Introduction

  • Given full evidence graphs, narrators still cite nonexistent cards, distort numbers, flip signs, or invent plausible facts.Source: §1 · p. 1
  • VERITYGATE introduces claim-level checks for evidence existence, entity allowlisting, number binding, and claim-type satisfaction.Source: §1 · pp. 1–2
  • The paired benchmark tests 900 instances at each r=0 and r=1 setting, covering 450 matched settings with three narrators.Grounded and ungrounded variants yield the 900 instances per setting.Source: §1 · p. 2
  • An inference-time repair pipeline returns failing claims to the LLM with verifier errors for one repair pass, without modifying the model or backend.Source: §1 · p. 2
  • A counterbalanced judge protocol records an 83% position-A rate, illustrating why prose-only preference cannot substitute for mechanical faithfulness verification.Source: §1 · p. 2
  • Credit-card portfolio optimization is the case study, using a MILP backend that provides structured evidence with audit-quality ground truth.Source: §1 · p. 2

2 A Four-Gate Verification Taxonomy

The taxonomy defines four structural checks for claims grounded in backend evidence. Across the evaluated settings, claim-type errors persist after repair and suggest a type-routing bottleneck.

2 A Four-Gate Verification Taxonomy

  • Four failure families account for essentially all observed failures, with each family corresponding to one verification gate.Source: §2 · p. 2
  • Gate 1 rejects cited evidence identifiers absent from backend output; at r=0, Sonnet has 1 error and GPT-4o-mini has 6 per 150 grounded instances.After one repair pass, the counts drop to 0 and 4, respectively.Source: §2 · p. 2
  • Gate 2 requires cited entity names to come from the deployment’s allowed entity set.The passage gives examples of paraphrased or invented card identifiers.Source: §2 · p. 2
  • Gate 3 requires cited numbers to exactly match canonical backend strings, while allowing small integers 0–3 by convention.At r=0, GPT-4o-mini has 388 number errors and Llama-70B has 182 across 150 grounded instances; Sonnet has 0 at r=0 and 1 at r=1.Source: §2 · p. 2
  • On Sonnet, Gate 4 errors decline only from 985 (r=0) to 841 (r=1) across 150 instances, and a second repair pass moves them no further.Across tested models and repair settings, this persistence suggests a type-routing bottleneck; broader model and domain studies are needed.Source: §2 · pp. 2–3
  • Figure 1 contrasts a FEE_JUSTIFICATION claim that passes all gates with a COMPARISON claim rejected for citing an unallowed entity.The traces are pedagogical, not exact serialized logs.Source: Figure 1 · p. 3

3 The VERITYGATE Framework

VERITYGATE is a transparent validation layer between a structured backend and an LLM narrator. It constructs an evidence graph, checks declared claims against it, and can repair or filter failing claims.

3 The VERITYGATE Framework

  • VERITYGATE operates between the deterministic backend and narrator without modifying either, and performs validation in three stages.Source: §3 · p. 3

3.1 Stage 1: Evidence Graph Construction

  • Stage 1 turns backend evidence blocks into graph nodes, connects related nodes, and maintains entity and number allowlists.The canonical graph also carries a SHA-256 digest for deterministic, auditable verification.Source: §3.1 · p. 3

3.2 Stage 2: Four-Gate Verification

  • Stage 2 checks declared claim fields against the graph using four structural gates, rejecting any claim that fails a gate.The verifier checks schema-level structure rather than prose-level faithfulness.Source: §3.2 · p. 3

3.3 Stage 3: Repair Loop and Permissive Emission

  • Stage 3 can return failing claims with specific errors for one repair pass, then retain the list with fewer failing claims and emit only verified claims.On Sonnet, a second pass has no statistically distinguishable gain; for Llama, one pass reduces shipped volume without a detectable survival gain.Source: §3.3 · p. 4

4 Evaluation

VERITYGATE is evaluated on paired grounded and ungrounded narrations from three LLMs, with and without one repair pass. Repair effects vary by model, while claim-type rules dominate the remaining failures at r=1; the judge diagnostic also reveals a strong position effect.

4 Evaluation

  • The evaluation reports claim-survival rate, shipped verified claims per instance, and judge-based usefulness preference as three distinct dimensions.Survival and shipped volume can move differently, so the paper treats both as co-primary metrics.Source: §4 · p. 4, §4.1 · p. 5, Table 3 · p. 7

4.1 Setup

  • The benchmark uses 50 parametric profiles and three goal types (150 instances); at each of r=0 and r=1, three narrators and two variants yield 900 instances and 450 matched grounded–ungrounded pairs.r=0 is the no-repair baseline, and r=1 uses one repair pass.Source: §4.1 · pp. 4–5

4.2 Main results

  • At r=1, Sonnet exceeded mini by +26.4 pts [+22.6, +30.1] in claim-survival rate.One-pass repair increased shipped claims by +0.14 for mini, while shipped claims fell by −0.71 for Llama and −0.47 for Sonnet.Source: §4.2 · p. 6, Table 3 · p. 7, Table 2 · p. 7
  • At r=1, Gate 4 covered 97.0%, 98.7%, and 100% of distinct failing claims for mini, Llama, and Sonnet, respectively.On Sonnet, the only Gate 3 failure overlapped a Gate 4 failure; Gates 1–3 were otherwise nearly eliminated.Source: §4.2 · p. 6, Figure 4 · p. 6

4.3 Counterbalanced Judge Analysis

  • Across 300 judge calls, slot A was chosen 83% of the time, while all three mean preference scores were below 0.5, favoring ungrounded prose.Because the judge lacked the evidence graph, these scores measure usefulness preference rather than faithfulness.Source: §4.3 · p. 7, Figure 5 · p. 8

4.4 Cost and deployment characteristics

  • At r=1, Sonnet costs ≈31× as much as Llama while shipping ≈1.86× as many verified claims per instance; verification sustains ≈88,000 checks/sec/core.The cost estimates are not independently reestimable from the supplement because the underlying provider token-count export was not preserved.Source: §4.4 · p. 8

5 Discussion and Lessons Learned

At r=1, Gate 4 remains the main source of failing claims, while sparse explanations and untested cross-domain transfer mark important deployment boundaries.

5 Discussion and Lessons Learned

  • At r=1, Gate 4 accounts for 841 distinct failing Sonnet claims, while Gates 1–3 have 0/0/1 errors.A second pass has no clear effect; across tested tiers, the authors interpret this as a type-routing bottleneck rather than a universal ceiling.Source: §5 · p. 8
  • At r=1 grounded, zero-verified-claim rates range from 24.7% for mini to 1.3% for Sonnet.The authors identify a minimum threshold, deterministic template fallback, or stronger narrator as deployment options.Source: §5 · p. 8
  • Medical and loan-risk mappings are proposed as examples, but cross-domain transfer is not empirically tested.The graph builders and Gate 4 rules are domain-specific.Source: §5 · p. 8

6 Related Work

Prior work evaluates structured-data and citation faithfulness through slot errors, table-to-text metrics, and citation checks. VERITYGATE adds a typed declaration/evidence contract and paired repair/filter evaluation, alongside work on other defenses and metrics.

6 Related Work

  • Slot-error, table-to-text, and citation methods address structured-data and citation faithfulness; VERITYGATE adds a typed declaration/evidence contract and paired repair/filter evaluation.The paper describes this as an added contract and evaluation, not a new set-membership operation.Source: §6 · p. 8
  • Other related work covers NLI and atomic-fact metrics, constrained syntax, repair, judge bias, and tool safety.The passage cites Es et al. (2024), Rashkin et al. (2023), Min et al. (2023), Gao et al. (2023a), Yue et al. (2023), and Chen et al. (2024) for NLI and atomic-fact metrics.Source: §6 · p. 8

Ethical Considerations

The work studies grounded explanations in a controlled credit-card portfolio benchmark, without production systems or end users. It reports parametric profiles and warns that its verified label signals schema-level consistency, not complete prose accuracy.

Ethical Considerations

  • Experiments used controlled local environments and a credit-card portfolio benchmark, with no production systems or end users.The generated claims received offline labels from three external raters on a small sample.Source: Ethical Considerations · p. 8
  • Benchmark profiles were generated parametrically, and no directly identifying personal data was processed.The prior backend validation referenced public-use BLS microdata, not personally identifiable records.Source: Ethical Considerations · p. 9
  • A “verified” label indicates schema-level consistency, not full prose faithfulness, so deployments must not present it as a guarantee of factual accuracy.The authors recommend surfacing this scope to users through product copy and/or audit logs.Source: Ethical Considerations · p. 9

Limitations

The evidence is bounded by three tested LLM backends, one application domain, and limited human and judge studies. The paper also leaves seed robustness, adversarial robustness, and comparisons with templates or neural faithfulness metrics untested.

Limitations

  • Evaluation covers three LLM backends, and the observed repair and Gate 4 patterns may not generalize to future models without replication.Both the pattern and absolute numbers require replication.Source: Limitations · p. 9
  • Profile generation uses a single seed, so the reported paired bootstrap confidence intervals do not assess seed robustness.The authors describe those intervals as sufficient for within-benchmark claims.Source: Limitations · p. 9
  • External rater studies use small stratified samples, and test agreement with verifier calls rather than independently re-deriving ground truth.Each study used three raters and 25 items; the authors call for larger studies with domain experts and broader coverage.Source: Limitations · p. 9
  • The judge analysis uses one model and is reported as a usefulness preference, not a faithfulness measure.Other judge models may show different position-bias magnitudes and absolute preferences.Source: Limitations · p. 9
  • Adversarial robustness is not evaluated; a model that knows the allowlist could pass verification while remaining semantically misleading.The stated threat model is unintentional fabrication, not adversarial attack.Source: Limitations · p. 9
  • All empirical results concern credit-card portfolio optimization with a MILP backend; medical and loan-risk mappings are not empirically tested.The authors identify empirical cross-domain transfer as the most important follow-up.Source: Limitations · p. 9
  • The study does not compare narrative quality or coverage with deterministic templates or run NLI, RAGAS, or ALCE scorers.Its results establish the structural audit-and-filter contract, not superiority over templates or complementarity to neural faithfulness metrics.Source: Limitations · p. 9

Appendix A. Independent Rater Validation

The rater studies found both agreement and systematic disagreement with the verifier: numeric string formatting and undeclared prose can separate prose judgments from structural checks. A separate Gate 4 study found majority agreement on 22 of 25 cases, while one rater applied a looser evidence-type standard.

Appendix A. Independent Rater Validation

  • On a 25-item Gates 1–3 sample, mean pairwise rater agreement was 74%, and majority-vote agreement with the verifier was low (κ = 0.02).The study excluded Gate 4 items because those require domain-specific structural judgment beyond the rubric.Source: Appendix A · p. 11
  • Four unanimous rater–verifier disagreements involved numerically equivalent strings such as 2400.0 and 2400.00 failing the verifier’s exact-string check.The authors identify numeric canonicalization as a future improvement.Source: Appendix A · pp. 11–12
  • One unanimous rater rejection passed verification because the claim included “general purchases” in prose but emitted empty cited fields.The verifier had no declared values to inspect in that case.Source: Appendix A · p. 12
  • All three raters rejected a claim with an unallowed number, while all three accepted a separate claim whose numerically correct values failed exact-string matching.The examples illustrate agreement on a clear Gate 3 violation and disagreement over formatting-equivalent values.Source: Appendix A · p. 12
  • 22 of 25 raters’ majority decisions agreed with Gate 4 rejections (88%), while mean per-rater agreement was 86%.The 25-item study sampled Gate 4 failures; chance-corrected κ values were interpreted cautiously because of skewed prevalence.Source: Appendix A · p. 12, Table 5 · p. 13

Appendix B. Rater Rubric

  • Raters used a literal rubric: check whether every claim entity and number appears in the evidence box, without outside knowledge or arithmetic.They could mark UNCLEAR when a claim was vague or a paraphrase was uncertain.Source: Appendix A · p. 11, Appendix B · p. 13

Appendix C. Claim Schema and Gate 4 Type Rules

The appendix specifies the claim fields and four structural checks, including domain-specific rules that link claim types to required evidence types. Its prompt templates supply evidence and allowed-value lists, while repair feedback returns verifier errors to the model.

Appendix C. Claim Schema and Gate 4 Type Rules

  • The six-field claim schema declares claim ID, type, prose, evidence IDs, entities, and numbers; Gates 1–3 validate the corresponding declared arrays.Gate 4 requires at least one cited evidence node of a claim-type-specific required type; CAP_SWITCH can require two distinct types when present.Source: Appendix C · p. 13, Table 6 · p. 14
  • Gate 4 failure messages most often involved COMPARISON missing WINNER_BY_CATEGORY and FEE_JUSTIFICATION missing FEE_BREAK_EVEN evidence.At claim level, 89 of 106 Sonnet CAP_SWITCH failures omitted CAP_HIT, 72 omitted ALLOCATION_SEGMENT, and 55 omitted both.Source: Appendix C · p. 13, Table 6 · p. 14
  • In a 27-instance replay, 138 of 139 grounded Gate 4 failures cited evidence, but generally not the required type.The cited required type was available in the graph in the great majority of cases; the prompt supplies node types but does not enumerate the type-to-required-evidence mapping.Source: Appendix C · p. 14
  • Table 7 counts verifier error messages rather than distinct failing claims, and CAP_SWITCH can produce up to three messages per claim.COMPARISON missing WINNER_BY_CATEGORY is the largest sub-mode on every model.Source: Table 7 · p. 14, Appendix C · p. 14

Appendix D. Prompt Templates and Sample Evidence Graph

  • Grounded prompts provide the full evidence graph and verbatim allowed-entity and allowed-number lists; ungrounded prompts provide only top-line figures and the portfolio card list.The prompt variants differ in supplied evidence, not instructions to hallucinate or be careful.Source: Appendix D · p. 15
  • Repair prompts append up to ten specific verifier errors to the original grounded prompt and request a corrected JSON claim list.The judge sees explanation prose and a goal/portfolio header, but not the evidence graph or verifier output, so its result is a usefulness diagnostic rather than a faithfulness measure.Source: Appendix D · p. 15

Appendix E. Pre-Registration Summary

The pre-registration specifies paired metrics, comparisons, and statistical procedures, alongside explicit non-claims. Later deployment and judge analyses were designated post-registration choices.

Appendix E. Pre-Registration Summary

  • The co-primary outcomes were claim survival and mean shipped verified claims per instance under permissive emission.The implemented claim-survival denominator counts parsed-and-retained claims.Source: Appendix E · p. 16
  • Repair effects were paired by profile, goal, and model; cross-model comparisons at r=1 and the Sonnet r=2 ablation were paired by profile and goal.The statistical procedure used 10,000 instance-cluster bootstrap iterations and paired 95% confidence intervals.Source: Appendix E · p. 16
  • A repair benefit was to be claimed only when the paired survival confidence interval excluded zero; the planned diminishing-returns rule used the Sonnet smoke r=2–r=1 interval.That interval crossed zero at N=9.Source: Appendix E · p. 16
  • The authors pre-registered no claim of state-of-the-art performance on open-domain factuality or RAG benchmarks, nor empirical cross-domain transfer.They also made no unqualified claim that grounding beats the ungrounded variant.Source: Appendix E · p. 16
  • The best-of-(r=0, r=1) deployment policy and judge sample were specified after registration; the judge is a usefulness diagnostic, not evidence that generation improves.The observed 1.78% schema-failed instance rate at r=1 is descriptive; the 5% stop rule concerned provider calls.Source: Appendix E · p. 16

Appendix F. Structured-Data and Citation-Metric Lineage

The paper positions VERITYGATE alongside prior structured-text evaluation and citation work, while distinguishing its typed declaration/evidence contract and repair-and-filter evaluation.

Appendix F. Structured-Data and Citation-Metric Lineage

  • VERITYGATE adds a typed declaration/evidence contract and repair-and-filter evaluation; neural-metric and relaxed-rule comparisons remain future work.The passage contrasts this scope with slot-error and semantic-control work, table-to-text metrics, and citation correctness/completeness evaluation.Source: Appendix F · p. 16

Related papers

Loading 2610.00833v1…