Source-linked AI summary
VERA-8B: Evidence-Grounded Audit Risk Reasoning from SEC Filings
Menghan Liu, Elynn Chen
TL;DR
Audit models need evidence-grounded, pre-enforcement judgments rather than plausible but ambiguous financial-language outputs. VERA-8B unifies evidence-constrained post-training, verification, abstention, uncertainty routing, and AuditBridge, and it leads evaluated baselines while producing structured, review-ready findings.
Problem
Audit reasoning requires exact, contextually valid filing evidence, while enforcement outcomes cannot directly label the evidence available before enforcement.
Method
VERA-8B combines structured SFT, verifier-guided adaptive GRPO, evidence verification, abstention, uncertainty routing, and AuditBridge under one evidence standard.
Results
VERA-8B leads all evaluated baselines, achieving 95% binary F1, 88% category micro-F1, 87% evidence-verified micro-F1, and 94% verified-positive recall with 3% unsupported claims.
Takeaways & Limitations
The system produces auditable, evidence-backed findings and routes ambiguous cases to review for practical audit workflows.
Takeaways & Limitations
The certification bound does not pass, so no deployment certificate is issued; certification remains fail-closed.
Abstract
from arXiv · showhide
Across audit applications, judgments must be supported by reasonable evidence. However, standard financial language models prioritize fluency over evidence. They are built for general financial reasoning and may produce plausible but ambiguous answers, creating a grounding gap that makes them unsuitable for audit work. We address this gap with VERA-8B, a new end-to-end audit reasoning system that identifies audit risks before enforcement actions occur. Constructing such a model raises several challenges, as no prior machine learning work targets pre-enforcement audit prediction. To our knowledge, we are the first to unify SFT and GRPO for evidence-grounded audit reasoning under one evidence standard, achieving performance that surpasses all evaluated baselines. Because auditing cannot tolerate unsupported claims, we introduce abstention and uncertainty qualification to defer uncertain or evidence-incomplete cases. Finally, we design an AuditBridge to ground model reasoning for practical audit work. It transforms raw filings into verified records and then into reviewer-ready reports, bridging finance and computation with broad generality. Together, these components produce auditable, review-ready outputs suitable for practical audit work.
1 Introduction
Audit reasoning requires filing evidence that is timely, contextualized, and tied to an audit mechanism, not merely plausible predictions or red-flag words. VERA-8B addresses this grounding gap by producing review-ready findings, abstaining when support is insufficient, and leading benchmark evaluations.
- Motivation: Audit evidence must connect an exact filing quote to its timing, context, and audit mechanism; lexical red flags alone are insufficient.Material-weakness, internal-control, going-concern, and accounting-estimate disclosures may describe current issues, remediated history, or boilerplate.
- Motivation: Enforcement outcomes identify issuer-periods for investigation but cannot serve as direct filing-level evidence labels for pre-enforcement prediction.The filing must be evaluated using information available before enforcement, and no enforcement action does not imply no audit risk.
- Approach: VERA-8B identifies future audit risks directly from filing evidence, explains each finding with exact support, and abstains when evidence is insufficient.The model transfers raw filing text into evidence-backed, review-ready audit findings.
- Results: 95% binary F1, 88% category micro-F1, 87% evidence-verified micro-F1, and 94% verified-positive recall establish VERA-8B as the benchmark leader.All test cases produced valid structured outputs, while unsupported claims were 3%.
- Contributions: The unified post-training framework uses one evidence contract across supervision, generation, verification, and refinement, combining structured SFT with verifier-guided adaptive GRPO.It also incorporates exact evidence verification, abstention, and uncertainty-qualified routing.
- Contributions: AuditBridge converts filings into verified JSON records and reviewer-ready reports, connecting model reasoning to downstream audit review.The pipeline runs from text to computation to review and is described as reusable for other evidence-constrained tasks.
2 Related Work
Prior work advances misstatement prediction, financial reasoning, selective prediction, and audit evidence discovery, but these strands leave evidence-constrained audit reasoning incomplete. This work combines temporally aligned supervision, an executable rulebook, audit-specific post-training, and selective abstention to make judgments verifiable.
- Misstatement Risk Detection Models: Misstatement-risk models use financial, firm-level, accounting, governance, market, audit, and 10-K language features to predict risk.Realistic evaluation must account for class rarity, chronological splits, and delayed discovery.
- LLM Post-Training and Selective Prediction: SFT learns structured response patterns, GRPO refines them through group-relative rewards, and calibration methods support uncertainty qualification.The related methods include LoRA, QLoRA, temperature scaling, selective classification, and conformal prediction.
- Financial LLMs and Task Specialization: Financial language models and task-specific systems improve reasoning over financial text, tables, and equations through targeted data and optimization.Examples include FinBERT, FinQA, Fin-R1, and Fino1.
- Audit Reasoning and Multi-Label Evidence: Audit research progresses from rule-based expert systems and explanation studies to opinion-text prediction and expert-guided cross-document fraud evidence discovery.AuditFraudBench evaluates joint reasoning across filings, restatements, financial figures, narratives, and enforcement mechanisms.
- This Work: The proposed framework combines temporally aligned enforcement supervision, an executable rulebook, evidence-aligned multi-label SFT, audit-specific GRPO, and selective abstention.Together, these components support verifiable audit judgments and deferral when evidence is insufficient.
3 Task Definition and Data Sources
The task treats each SEC filing as an evidence-grounded audit-reasoning instance with multi-label risks, supporting evidence, explanations, and abstention. Its data sources separate pre-enforcement filings, ex-post enforcement outcomes, and public audit knowledge, while preserving the distinction between enforcement status and admissible evidence.
- 3.1 Audit-Risk Reasoning Task: Each filing is mapped to an output containing audit-risk categories, supporting evidence, an evidence-to-risk explanation, and an abstention decision.The task is formulated over individual SEC filings.
- 3.1 Audit-Risk Reasoning Task: The risk vector has nine non-exclusive categories, and one observation may contain zero, one, or multiple categories.Categories include internal control, restatement, revenue recognition, fraud or misconduct, going concern, covenant or default, disclosure opacity, auditor change or disagreement, and accounting estimates.
- 3.2 Data Sources: The framework separates pre-enforcement filing information, ex-post enforcement outcomes, and public audit knowledge into distinct source roles.These source families are summarized in Table 1.
- 3.2 Data Sources: Enforcement status does not guarantee admissible pre-enforcement evidence, and its absence does not imply that a filing lacks audit-risk evidence.This asymmetry creates evidence-supported positives and hard abstention cases.
4 Methodology
The methodology separates retrospective enforcement alignment from admissible filing evidence, then trains and routes an audit reasoner under one evidence contract. VERA-8B combines structured supervision, verifier-guided refinement, and evidence-qualified deferral.
- Outcome-Aligned Dataset Construction: Stage 1 links filings to later enforcement outcomes using issuer identity, fiscal period, and temporal precedence, while retaining uncertain matches for review or sensitivity analysis.Enforcement links identify observations for investigation but are not treated as filing-level evidence labels.
- Executable Rulebook Construction: An executable, versioned JSON rulebook maps filing evidence to audit-risk categories using sources, filing sections, triggers, verification conditions, exclusions, and minimum evidence strength.The rulebook combines SEC filing sections, PCAOB concepts, and recurring mechanisms from historical enforcement actions, while allowing external expert rules without weakening the evidence standard.
- Executable Rulebook Construction: Evidence gates admit a filing span only when its quotation, context, mechanism support, and temporal conditions pass; failed candidates become hard negatives and unresolved conflicts go to review.Historical-only statements and completed remediation are excluded from admission.
- Evidence-Closed Contract Learning: Training targets require every positive category to pair with verified evidence and a mechanism-level rationale, assigning abstention when no admissible category–evidence pair exists.Thus, a later enforcement link alone cannot create a positive SFT target.
- Evidence-Closed Audit Alignment: Stage 2 applies structured SFT and adaptive GRPO under a shared verifier that checks schema validity, taxonomy consistency, evidence identifiers, literal phrase containment, and abstention consistency.Adaptive reward weighting shifts training pressure toward unresolved objectives, while a KL penalty and validation gate prevent unsupported claims from being traded for recall.
- Evidence-Qualified Selective Routing: Split-conformal prediction sets combined with the training verifier route evidence-admissible positives, valid negatives, or uncertain and evidence-incomplete cases to review.Only evidence-admissible predictions are automated; ambiguous, malformed, or evidence-incomplete outputs are deferred.
5 Experiment Results
Under a locked issuer-disjoint frozen-test protocol, VERA-8B leads the full audit task by combining risk reasoning, admissible evidence, structured outputs, and selective review.
- Evaluation protocol: 1,960 verified filings are split by issuer into 1,352 training, 298 validation, and 310 frozen-test observations.All inputs are restricted to pre-enforcement filing text, with no issuer shared across splits.
- Evaluation protocol: All models receive identical filing inputs and output schemas, with deterministic parsing and evidence verification under one frozen protocol.The frozen test set is evaluated only once, and outputs are scored without post-hoc repair.
- Benchmark results: 45% higher binary F1 and 83% higher verified-positive recall accompany a reduction in unverified claim rate from 16.0% to 3.1% versus DeepSeek-V4-Pro.Against the strongest finance-specific baseline, category micro-F1 rises from .648 to .878.
- Benchmark results: 95.3% binary F1 and 87.1% EV-Micro-F1 are achieved with valid JSON for all 310 frozen-test observations.The end-to-end score jointly covers risk detection, multi-label reasoning, admissible evidence, and abstention.
- Post-training: Structured SFT raises binary F1 from 13.0% to 94.7% and EV-Micro-F1 from 34.3% to 86.4% over the base Llama-3.1-8B model.It also yields 100% valid JSON and reduces unverified claim rate from 65.2% to 3.2%.
- Post-training: Adaptive GRPO raises verified-positive recall from 91.5% to 93.6% while false negatives fall from five to three, with JSON validity at 100% and UCR unchanged.The paired bootstrap intervals include zero and the exact McNemar test gives p=1.0.
- Selective review: Uncertainty routing automatically handles 258 of 310 filings, sends 52 to manual review, and gives automatic positives zero UCR.All 182 automatic negatives achieve 100% negative predictive value, with selective risk of 1.94%.
- AuditBridge: AuditBridge stores each decision as machine-readable JSON and renders the verified record as a reviewer-ready report containing evidence, reasoning, routing, and uncertainty.The two output forms support downstream financial processing and audit review while preserving traceability to filing evidence.
6 Conclusion
VERA-8B turns pre-enforcement SEC filings into evidence-grounded audit findings through unified post-training, selective review, and verified output formats. It leads evaluated baselines while maintaining structured outputs and low unsupported-claim rates.
- Conclusion: VERA-8B identifies audit risks, cites admissible evidence, abstains when support is insufficient, and refines audit-specific errors under one evidence standard.The framework integrates these behaviors into an end-to-end audit-specific reasoner.
- Conclusion: VERA-8B leads all evaluated baselines while uncertainty routing directs ambiguous cases to review and AuditBridge connects verified outputs to processing and audit review.The conclusion presents the system as a practical foundation for scalable, evidence-grounded audit workflows.