PaperlayerView exact revision on arXiv

Source-linked AI summary

A decision-basis contract for auditable LLM-assisted medical billing verification: deterministic rules, verbatim evidence, and fail-closed abstention

Jan Hölter, Kevin Geis, Benjamin Raab, Boris Bauke

arXiv:2609.12156v1cs.SEcs.AI

TL;DR

Medical billing verification must reconcile formal fee-catalog rules with free-text documentation, while making model assessments and abstentions inspectable. This paper proposes a two-layer decision-basis contract and evaluates it on synthetic cases, finding no consistent outcome-agreement advantage over the baseline but improved missing-information identification with explicit documentation requirements.

  • Problem

    Billing verification must determine whether claims comply with applicable fee-catalog rules and whether documentation supports the claimed services, while making model judgments and abstention reasons traceable.

    Source: §1 · p. 1

  • Method

    A decision-basis contract separates deterministic checks of versioned catalog rules from LLM documentation assessment and requires verbatim evidence for support or contradiction.

    Source: §1 · pp. 1–2, §3.1 · p. 3

  • Results

    Outcome agreement showed no consistent advantage over the baseline, while explicit documentation requirements improved missing-information identification across all four models.

    Source: §6 · p. 7, §5 · p. 7

  • Takeaways & Limitations

    Explicit decision records make rule findings, documentation judgments, and abstention reasons inspectable in this proof of concept.

    Source: §5 · p. 6, §6 · p. 7

    The synthetic consensus acceptance suite lacks independent annotation and a sample of billing practice; real catalogs, independently annotated data, and reviewer studies are needed to assess practical audit support.

    Source: §5 · p. 7

Abstract

from arXiv · show

This work presents a proof of concept for auditable LLM-assisted medical billing verification based on a decision-basis contract. The contract separates deterministic checks of versioned fee-catalog rules from LLM-based assessment of free-text documentation. The deterministic layer resolves the applicable catalog release and checks code availability, quantity limits, and exclusions. The semantic layer classifies each claimed item as supported, contradicted, or missing required information. Support and contradiction require a verbatim evidence span; unavailable rule context, unsuccessful assessment, or missing required evidence prevents support through fail-closed abstention. We evaluated four locally run open-weight models on a synthetic catalog and 36 curated cases under the contract, an ablation without explicit documentation requirements, and an end-to-end baseline. Outcome agreement varied across models and showed no consistent advantage over the baseline. Explicit documentation requirements improved identification of missing information for all four models. The evidence gate also exposed cases in which correct raw judgments lacked valid evidence and were converted to incomplete decision-basis entries. The results show how explicit decision records can make rule findings, documentation judgments, and abstention reasons inspectable. Evaluation on real catalogs, independently annotated documentation, and with human reviewers is required to assess practical value.

1 Introduction

Medical billing verification combines checks against formal catalog rules with judgments about whether free-text documentation supports claimed services. This paper proposes an auditable two-layer contract and evaluates its decision records across four locally run models.

1 Introduction

  • The contract separates deterministic catalog checks from LLM assessment of whether free-text documentation supports each claimed item.The deterministic layer resolves the applicable release and checks quantity limits and exclusions; the semantic layer classifies items as supported, contradicted, or missing required information.Source: §1 · pp. 1–2
  • Support and contradiction require verbatim documentation evidence, while unavailable context or technical and evidence failures prevent support through fail-closed abstention.The decision basis records rule findings and effective documentation judgments, with a fixed precedence determining the final outcome.Source: §1 · p. 2
  • Outcome agreement was higher under the system than the end-to-end baseline for some models and lower for others.The evaluation also examines how evidence requirements and outcome derivation affect the final decision.Source: §1 · p. 2
  • The evaluation compares the contract with an ablation lacking explicit documentation requirements and an end-to-end baseline, using a synthetic catalog and reference dataset.The authors present the work as a proof of concept for making rule findings, documentation judgments, and abstention reasons inspectable.Source: §1 · p. 2

2 Related work

Prior medical coding systems support formal consistency, while semantic conflicts and model-generated errors motivate verification-oriented and human-supervised approaches.

2 Related work

  • Rule-based coding systems ensure formal consistency but address semantic conflicts only to a limited extent, while stand-alone LLM coders can produce imprecise or fabricated codes.An assistive coding trial reduced median coding time by 46% for longer texts without significantly improving accuracy.Source: §2 · p. 2
  • Verification and neuro-symbolic approaches include reducing hierarchically close misassignments, auditable LLM-rule chains, constrained decoding with conflict rules, and evidence-bound review trees.Fine-tuned models for German GO¨A radiology coding reached competitive multi-label performance, but manual review was still recommended.Source: §2 · p. 2
  • Retrieval-augmented generation is being explored to ground medical outputs, but its evaluation practices remain insufficiently standardized.Related concepts include claim labels of Supported, Refuted, or NotEnoughInfo and fail-secure behavior adapted as fail-closed alignment [12].Source: §2 · p. 2

3 Methods

The methods define a two-layer decision contract, construct a synthetic catalog and acceptance suite, and compare contract-based assessments with an ablation and an end-to-end baseline.

3.1 Problem and decision contract

  • The decision contract assigns structured release, code, quantity, and exclusion checks to a deterministic layer and documentation judgments to an assessor.Verbatim evidence is retained for support or contradiction; unavailable assessments or invalid evidence create incomplete basis entries, and outcome precedence derives the case result.Source: §3.1 · pp. 2–3

3.2 Synthetic catalog and reference dataset

  • The synthetic catalog models the German GO¨A schedule with 14 codes across five groups and two releases, while the 36-case suite exercises selected contract behaviors.The cases include 83 claimed items and reference labels for 79 semantic assessments; their LLM-drafted scenarios were jointly curated by two authors.Source: §3.2 · pp. 4–5, Table 2 · p. 4

3.3 Experimental design and evaluation

  • Each model was evaluated under the system, a system ablation without documentation-requirement fields, and a baseline that received the case and full catalog in one request.Evaluation compared exact outcome matches and false acceptances across 36 cases, raw verdict accuracy on 14 missing-attribute items, evidence-gate downgrades, and technical failures.Source: §3.3 · p. 5

4 Results

Outcome agreement varied by model and condition, while explicit documentation requirements improved raw judgments on missing-attribute items. Evidence-gate interventions could also turn correct raw judgments into incomplete decision-basis entries.

4 Results

  • For medgemma:27b, agreement rose from 19 to 22 cases and false acceptances fell from 14 to 5; no condition led agreement for every model.Table 3 reports correct cases and false acceptances across all recorded cases, including outcomes affected by unsuccessful assessments.Source: §4 · p. 6, Table 3 · p. 6
  • On 14 REQUIRED ATTRIBUTE MISSING items, removing documentation requirements reduced correct raw verdicts from 13 to 7 (gemma4:31b), 5 to 0 (medgemma:27b), 14 to 11 (qwen3:14b), and 12 to 11 (qwen3:4b).Counts use the same reference items before the evidence gate, retaining unsuccessful assessments in the denominator. Agreement was higher with explicit requirements on this subset, though gemma4:31b and qwen3:4b had higher overall agreement under system-ablation.Source: §4 · p. 6
  • In case b02, the system returned NOT EVALUABLE for an uncatalogued claim even though the baseline returned SUPPORTED; both supported the wound-treatment item.The system recorded the missing rule context in the decision basis.Source: §4 · p. 6
  • Across system and system-ablation, the evidence gate changed 21 item verdicts to DECISION BASIS INCOMPLETE, including 19 that matched the reference before the gate.In case c01, a correct contradiction judgment without a valid evidence span became NOT EVALUABLE rather than the expected NOT SUPPORTED.Source: §4 · p. 6

5 Discussion

The discussion presents the contract as an inspectable decision basis, but finds no general accuracy advantage and leaves practical reviewer value untested. It identifies wording, evidence-gate, and evaluation-scope limits that qualify the findings.

5 Discussion

  • The contract makes rule findings, documentation judgments, and case outcomes inspectable, but agreement improved for some models and declined for others.Benefits to human reviewers remain to be evaluated.Source: §5 · pp. 6–7
  • The contract follows a hybrid extraction and constrained-coding trend, specifying deterministic checks, model-delegated checks, and how information gaps affect outcomes.This provides a common basis for examining successful and unsuccessful decisions.Source: §5 · p. 7
  • The comparison changes task allocation, context, and response protocol together, so it reflects complete approaches rather than isolated components.The baseline comparison used a compact corpus of 28 entries across two releases.Source: §5 · p. 7
  • The catalog must be available, correctly structured, and versioned; constructing it from a real rule corpus remains separate work.The compact baseline comparison leaves performance with a much larger corpus an empirical question.Source: §5 · p. 7
  • Explicit requirements helped all four models identify missing information, but ambiguity about whether listed attributes are jointly required or illustrative may affect both assessment and reference judgments.The authors call for clarifying requirement wording.Source: §5 · p. 7
  • The evidence gate can reject correct judgments lacking verbatim spans, and literal matching does not establish semantic relevance or assessment correctness.Even a single matching letter or digit can pass the check.Source: §5 · p. 7
  • The short German cases were LLM-drafted and jointly curated without independent annotation, forming a consensus suite rather than a sample of billing practice.The authors call for independently annotated practice data, larger rule corpora, and reviewer-use studies.Source: §5 · p. 7

6 Conclusion

The decision-basis contract makes LLM-assisted billing verification inspectable by separating deterministic rules from semantic judgments and deriving outcomes from recorded evidence. In the synthetic evaluation, agreement had no consistent advantage over the baseline, while explicit documentation requirements improved missing-information identification across all four models.

6 Conclusion

  • Outcome agreement showed no consistent advantage over the baseline, while explicit documentation requirements improved missing-information identification across all four models.The evaluation was a synthetic acceptance suite.Source: §6 · p. 7
  • Verbatim evidence requirements and recorded assessment failures make decisions traceable, but can reject correct raw judgments that lack valid evidence.Outcome derivation also enforces consistency in the resulting decisions.Source: §6 · p. 7
  • The contract separates deterministic rules from semantic judgments and derives outcomes from their recorded decision basis.The record supports inspection of rule findings, documentation judgments, and abstention reasons.Source: §6 · p. 7
  • Evaluation with real catalogs and human reviewers is needed to determine practical utility.The reported proof of concept used a synthetic catalog and curated cases.Source: §6 · p. 7

Declarations

The declarations report no competing interests or specific grant funding and disclose AI-assisted writing and coding tools used in preparing the work.

Conflict of interest/Competing interests

Funding

  • The research received no specific grant from funding agencies in the public, commercial, or not-for-profit sectors.Source: Funding · p. 7

Generative AI and AI-assisted technologies in the writing process

  • The authors used DeepL Write, Claude 5 Fable, and GPT-5.6 Sol for writing, and AI coding assistance including dataset case generation, then reviewed and edited the work.The authors state that they take full responsibility for the published content.Source: Generative AI and AI-assisted… · p. 8

Appendix A System prompts and schema

The appendix prompts distinguish a full-case baseline assessment from an assessor restricted to semantic documentation judgments. They specify release checks, verdict and evidence requirements, and final outcome formatting.

Appendix A System prompts and schema

  • The assessor selects support, contradiction, or missing required attributes; support and contradiction require short verbatim evidence spans, while missing verdicts identify absent attributes.The assessor does not decide quantity limits or exclusions, which are checked separately.Source: Appendix A · p. 8
  • The baseline prompt supplies the full case, claimed items, and catalog releases, and treats anything other than exactly one valid release as not evaluable.Release validity uses a start-inclusive, end-exclusive interval, with no upper bound when valid_to is null.Source: Appendix A · p. 9
  • For known codes, the baseline checks maximum quantities and exclusions, then assesses documentation against applicable rules, including total claimed quantities across repeated positions.It treats absent required attributes as missing and does not count one documented service more than once toward the claimed total.Source: Appendix A · p. 9
  • The baseline derives one case outcome using a fixed precedence among rule violations, unresolved context, and insufficient documentation, then returns item and overall explanations in the prescribed JSON order.Source: Appendix A · p. 9

Appendix B Synthetic catalog and reference cases

The appendix supplies a synthetic two-release catalog and 36 reference cases with verbatim documentation, claims, and expected verdicts. The cases exercise varied catalog requirements and distinguish reference expectations from observed model outputs.

Appendix B Synthetic catalog and reference cases

  • The synthetic catalog has 2024 and 2025 releases; all fields are shared except two maximum-count values, and it provides no legal basis for billing.The 2024 release ends before 2025-01-01; the 2025 release has no upper validity bound.Source: Table B1 · p. 9
  • Catalog entries cover symptom-linked and comprehensive examinations, venous blood draws, injections and infusions, wound care, consultations, and organ ultrasound, with required attributes varying by service.Examples include specimen type and venous route for blood draws, wound type and treatment for wound care, and named organs for ultrasound.Source: Appendix B · p. 10
  • The 36-case table preserves verbatim documentation and ordered claims, labels item support, contradiction, or missing information, and records rule bases and case outcomes as reference expectations.A dash indicates no semantic label or applicable release, not an additional verdict.Source: Table B2 · p. 11
  • In one ultrasound case, the liver is documented as examined while other organs are explicitly excluded; the reference labels the first-organ claim supported and the additional-organ claim contradicted.Source: Appendix B · p. 12
  • For repeated I01 claims, both positions receive contradiction labels when two services are claimed but only one is documented and further services are denied; this does not mean no blood sampling occurred.Source: Appendix B · p. 13
Loading 2609.12156v1…