Conflicted Records: Typed Adversarial Contracts for Auditable LLM Output Quality Enforcement

1 Introduction

• LLM outputs are persisted in production information systems, where operators need to know why constraints pass or fail and which spans triggered verdicts.The records are treated as data in search indexes, knowledge bases, recommendation feeds, and customer-facing systems.Source: §1 · pp. 1–2
• A record can pair a grounded fact with a hallucination or supported reasoning with an unsupported generalization, requiring opposite judgments about co-located text.Classical cleaning and a single LLM judge cannot reliably resolve this conflict, according to the paper.Source: §1 · p. 2
• The paper asks whether deterministic checks over structured inter-agent records catch defects missed by referential checks and support auditing without another model call.It also evaluates whether deliberation transfers across quality domains.Source: §1 · p. 2
• BiasGuard-D combines adversarial span-level claims, a multi-family quorum, and a deterministic invariant to validate verdicts and retain their provenance.Contradictory verdicts are quarantined rather than auto-repaired; the system’s contributions also include an ablation and cross-domain portability evidence.Source: §1 · p. 2
• The figure contrasts a conflicted input requiring opposite span actions with BiasGuard-D’s typed per-span instructions and provenance trail.A single LLM judge returns no verdict for the depicted input.Source: Figure 1 · p. 3

Constraint-based data quality and repair

• Classical data-quality methods check integrity constraints and repair violations, but cannot select a canonical repair when co-located spans require removal and preservation.BiasGuard-D quarantines verdicts that fail semantic consistency rather than auto-repairing them.Source: §2 · pp. 3–4

Quality enforcement on LLM outputs

• Retrieval-augmented, self-refining, and training-time approaches lack a way to verify that conflicting spans were resolved rather than suppressed; BiasGuard-D uses typed runtime records and deterministic verification.The related work also describes multi-agent debate as structured disagreement without typed formats or deterministic verification.Source: §2 · p. 4

Typed adversarial contracts and provenance

• Typed messaging and formal contracts provide inter-agent message-passing machinery, but the paper applies it to adversarial quality enforcement with deterministic checks across constraint types.Fixed schemas make instruction lineage queryable without re-running models.Source: §2 · p. 4

3.1 The Problem

• Records requiring both preservation and removal are decomposed into spans whose roles and judges’ assignments must be checked for consistency before downstream action.The intended operator use also needs span-level approval or removal and provenance linking instructions to source spans.Source: §3.1 · p. 4

3.2 The Data Model

• An AtomicClaim is an identifiable assertion about a text span, carrying a role-prefixed identifier, claim category, optional provenance identifiers, and confidence.Stable schemas and identifiers let the invariant operate deterministically without analyzing natural-language content.Source: §3.2 · p. 5

3.3 Pipeline Architecture

• Two adversarial roles exchange typed documents over bounded rounds, then three arbiters from distinct families vote by per-claim majority before the invariant check.The pipeline has five stages; a same-family auditor acts as a high-precision, lower-recall triage gate, while the HaluEval condition adds a cross-family auditor to raise recall.Source: §3.3 · pp. 5–7, Figure 2 · p. 8

3.4 The Combined Consistency Invariant

• The combined invariant checks that upheld claims have corresponding instructions and rejects near-identical spans simultaneously upheld for removal and preservation.Both components run in O(n) time without an LLM call; the semantic check catches contradictions that referential coverage passes.Source: §3.4 · pp. 7–8

4.1 Experimental Setup

• The evaluation tests invariant value, record-level diagnostic value, and transfer using BBQ Age, BBQ Disability, and HaluEval QA conditions.HaluEval uses disjoint hallucinated-answer and right-answer slices; the latter measures whether the pipeline damages already-correct records.Source: §4 · p. 9, §4.1 · p. 9

4 Evaluation

• The study frames its evaluation around whether semantic checks add value, stored records diagnose coordination, and deliberation transfers across quality domains.Both benchmarks label constraint failures on fixed-schema records.Source: §4 · p. 9, §4.1 · p. 9

4.2 Baseline Comparison and the Cost of Suppression

• On BBQ Age’s 115 ambiguous-arm items, CoT routed 108 answers to the neutral option regardless of content, while BiasGuard-D produced a smaller raw reduction (−11.3pp).SDP raised neutral responses from 30.4% under ZS to 65.2%, with zero within-item classification changes.Source: §4.2 · p. 10, Table 3 · p. 10

4.3 Invariant Analysis: Catching What Referential Coverage Misses

• Across datasets, semantic violations occurred on records that satisfied the referential invariant, showing that the semantic check catches defects referential checks miss.On BBQ Disability's identical 36 delegated items, semantic violation rates were 44.4% for SR-Med and 16.7% for P2P; the threshold was robust across [0.70, 0.85], shifting only 1.3pp at 0.90.Source: §4.3 · pp. 10–11, Table 4 · p. 11

4.4 Schema-Level Coordination as Diagnostic Signal

• On BBQ Disability, P2P and SR-Med had invariant-satisfaction rates of 83.3% and 55.6%, respectively, despite identical inter-Arbiter agreement of 0.57.HaluEval agreement was 0.98, compared with 0.49–0.57 on BBQ; the text attributes the contrast to objective grounding versus subjective stereotype categories.Source: §4.4 · p. 11

4.5 Cross-Domain Transfer

• 100% of hallucinations were removed (43/43), while right-answer match rose from 12.2% to 61.2% on hallucinated inputs.The deliberation layer transferred with role-framing changes; maintaining recall on objective hallucinations required a cross-family second auditor at triage.Source: Table 6 · p. 12, §4.5 · p. 12

5.1 Deployment Use Cases and Monitoring Signals

• The architecture is proposed for moderation, knowledge-base ingestion, and privacy-versus-utility release, where opposing span-level actions can be surfaced as auditable records.In these examples, incompatible actions include removal versus preservation and asserting versus retracting a fact.Source: §5.1 · p. 13
• On BBQ Disability, the iterative loop reduces quarantine from 44.4% under SR-Med to 16.7% under P2P, resolving roughly two-thirds of conflicts.A high FB% can still indicate a genuine unresolved conflict for which conservative fallback is appropriate.Source: §5.1 · p. 13
• Agreement rate, semantic violation rate, and claim-count asymmetry can be monitored over rolling windows without an additional classifier.The paper relates this inspectability to NIST AI RMF and the EU AI Act.Source: §5.1 · p. 13

5.2 Limitations

• HaluEval hallucination-removal labels rely on an LLM judge subject to the biases the architecture targets; a strict prompt mitigates, but does not eliminate, this concern.The authors state that a fully independent human judge would eliminate the concern.Source: §5.2 · p. 13
• With OR delegation, 19 of 24 false delegations on correct inputs occur when one auditor flags and the other passes.The delegation strategy is a tunable threshold whose trade-off is left for future work.Source: §5.2 · p. 13
• The Jaccard threshold is untested on paraphrase-rich corpora, while HaluEval hallucination removal depends on Arbiter capability.The paper notes that a denser paraphrase space could raise false positives; free-tier Arbiter replacements performed substantially worse on HaluEval.Source: §5.2 · pp. 13–14

5.3 Future Work

• Requiring both auditors to agree before escalation could reduce false delegations, but routes some true hallucinations through the pass-through path; three-auditor voting adds cost.The authors identify systematic evaluation of this trade-off as the nearest-term engineering extension.Source: §5.3 · p. 14
• Future work would test portability beyond two domains, assess Jaccard alternatives for paraphrase-rich data, replace judge labels with human annotation, and seek a free-tier Arbiter.The free-tier Arbiter goal is to match Claude Sonnet-level performance on objective hallucination tasks.Source: §5.3 · p. 14

• LLM-generated records can contain both valid and invalid material under one quality rule, and resolving that conflict requires deliberation rather than a threshold check.Source: §6 · p. 14
• The semantic component fires on verdicts referential checks miss, while removing the iterative loop drops invariant satisfaction by nearly 30 percentage points on the BBQ Disability ablation.Source: §6 · p. 14
• Two adversarial roles from distinct model families exchange typed claims, a three-family quorum adjudicates, and a combined invariant validates the result.Source: §6 · p. 14
• Deliberation transfers from stereotype to hallucination remediation with only role-framing changes, raising right-answer recovery by 49 points; the triage gate remains domain-sensitive.Source: §6 · p. 15
• The portable template assigns domain-appropriate claim types and opposing-bias advocate roles while retaining the combined invariant.The conclusion frames this as extending data-management discipline to LLM-generated records where inspectability is a deployment requirement.Source: §6 · p. 15