Source-linked AI summary
FinixDoc: Rethinking Financial Document Parsing Beyond Saturated Benchmarks
Hang Wang, Jin Zhang, Guoliang Xu, Pengyue Lu, Yao Li, Zijiao Zhang, Tianyu Huang, Weiqi Xiong, Yulong Wang, Chuqiao Lu, Wenkang Huang, Kai Yang, Yadong Li, Hui Li, Xingzhong Xu, Xiao Xu
TL;DR
Real-world financial document parsing demands accuracy, structural consistency, traceability, and verifiability, yet conventional benchmarks may not reflect deployment challenges. FinixDoc addresses this gap with an agentic system, a capability matrix, domain-adapted training, and realistic evaluation, achieving state-of-the-art performance among major open-source models on financial-domain benchmarks.
Problem
Real financial documents vary in visual quality and scale, while conventional benchmarks may not capture the challenges of low-quality, structurally complex, and extremely large documents in deployment.
Method
FinixDoc combines a domain-adapted FinixDoc-VL parser with a Document Parsing Capability Matrix, dual-path contrastive objectives, multi-model collaborative inference, and a human-in-the-loop Data Factory pipeline.
Results
FinixDoc-VL achieves state-of-the-art performance among major open-source models on the introduced financial-domain benchmark, with especially large gains in the Low-Quality and Underexplored Large-Scale Zones.
Takeaways & Limitations
Financial document parsing should be evaluated by stability, robustness, and usability on low-quality, high-complexity, and large-scale documents rather than conventional benchmarks alone.
Takeaways & Limitations
Specialized document models’ gains are concentrated on clean, regular distributions, while camera-captured documents remain underrepresented in training and evaluation.
Abstract
from arXiv · showhide
Financial document parsing requires accuracy, structural consistency, and verifiability that current benchmarks often fail to reflect. We present FinixDoc, an end-to-end agentic parsing system for real-world financial documents, with FinixDoc-VL, a 4B-scale vision-language model built on Qwen3-VL-4B, as its core parser. To characterize the gap between benchmark and deployment performance, we introduce a Document Parsing Capability Matrix organized along two practical axes: visual quality and document scale. Guided by this matrix, FinixDoc-VL is trained with a domain-adapted recipe combining homoglyph-aware contrastive learning and multi-stage reinforcement learning with composite domain-specific rewards. To better leverage our accumulated advantage in low-quality financial-document data and support large-scale, high-quality data production, we further build a human-in-the-loop Data Factory pipeline with confidence-aware expert review. For evaluation, we construct FinixDocBench, a financial-domain evaluation suite covering digital-native, camera-captured, ultra-large-page, and internal-workflow scenarios, with a compliance-reviewed subset released alongside this technical report. On its main subsets, FinixDoc-VL achieves the highest overall score (81.43) among evaluated baselines, outperforming the next-best open-source model by 5.13 points, with the largest gains on internal financial workflows (FinixInner: 84.08 vs. 78.73).
1 Introduction
Financial document parsing must meet stricter requirements than conventional benchmarks capture, particularly for heterogeneous, low-quality, structurally complex, and large-scale documents. FinixDoc addresses this gap with a capability matrix, domain-adapted FinixDoc-VL training, a human-in-the-loop Data Factory, and deployment-oriented evaluation.
- Motivation: Financial workflows process heterogeneous documents whose outputs require accuracy, structural consistency, traceability, and verifiability.
- Background: Modern parsing unifies text recognition, layout analysis, table recovery, and reading-order reconstruction into structured JSON or full-page Markdown.
- Benchmark Gap: Strong benchmark performance can coexist with weak robustness, structural inconsistencies, and failures on long documents or large images in production.
- Capability Matrix: The Document Parsing Capability Matrix organizes failure patterns along document quality and document scale, distinguishing benchmark-converged, low-quality, large-scale, and ambiguous-unrecoverable zones.
- Capability Boundaries: Extremely poor visual quality can make outputs speculative and ungrounded, motivating hallucination suppression and a better-omission-than-error principle.
- FinixDoc: FinixDoc combines a Qwen3-VL-4B-based FinixDoc-VL parser with homoglyph-based contrastive learning, reinforcement learning, a Data Factory, and realistic financial evaluation.
2 FinixDoc
FinixDoc combines a conditionally tool-using parsing agent with FinixDoc-VL, a Qwen3-VL-4B-based model adapted for financial documents. Its training targets low-quality financial inputs through contrastive representation learning and domain-specific reinforcement learning.
- Overall Architecture: FinixDoc dynamically selects preprocessing tools based on document modality, page scale, and image quality before parsing with FinixDoc-VL.The system supports heterogeneous inputs and produces page-aligned JSON or Markdown outputs.
- Document Preprocessing Tool Layer: Ultra-large pages are partitioned into manageable regions and then recombined into unified outputs that preserve full-page boundaries.This split-then-merge strategy addresses pages exceeding the model’s effective visual capacity.
- FinixDoc-VL: FinixDoc-VL extends Qwen3-VL-4B with visual representation adaptation and business-aligned decoder optimization for low-quality financial documents.The model retains a lightweight general-purpose backbone while targeting financial recognition, layout comprehension, and structured generation.
- Financial Homoglyph Vocabulary and Negative Sample Construction: Financial homoglyph hard negatives train dual-path contrastive objectives over visual and multimodal generative representations alongside supervised fine-tuning.The hard negatives model realistic recognition errors rather than relying on random perturbations or generic negative sampling.
- Reinforcement Learning Fine-Tuning with Domain-Specific Rewards: Domain-specific reinforcement learning jointly optimizes JSON and Markdown generation using task-dependent rewards for text fidelity, detection quality, and reading-order consistency.GRPO alignment follows multi-stage sub-capability calibration, with reward components selected according to output format.
3 Data Factory: A Human-in-the-Loop Production Pipeline for Financial Documents
FinixDoc’s Data Factory addresses domain specificity, document-quality variation, massive scale, and costly manual annotation through a human-in-the-loop pipeline. It combines multi-model fusion, heuristic page scoring, cascaded refinement, traceability, and quality-aware selection to produce validated in-domain data.
- Motivation: Financial data construction must handle specialized terminology and layouts, heterogeneous document quality, and millions of documents and tens of millions of pages.These constraints make large-scale manual annotation prohibitively expensive.
- Pipeline Overview: The Data Factory integrates model diversity, multi-stage refinement, and downstream quality feedback into an AI-driven human-in-the-loop pipeline.The pipeline is designed to address the three data-construction challenges in a unified closed loop.
- Stage 1: Multi-Model Inference: Three models jointly generate candidate annotations, using FinixDoc-VL as the schema-aligned anchor and aggregating category, spatial-overlap, text-similarity, confidence, and agreement signals.PP-DocLayoutV3 supplies layout detection, while Qwen3-VL-235B-A22B-Instruct adds broader semantic coverage.
- Multi-Dimensional Scoring: Page complexity combines region count, density, overlap, and layout-type diversity, while semantic richness combines OCR volume, financial-term density, and content diversity.The resulting page-level statistics support post-hoc selection and four-quadrant stratified sampling.
- Stage 2: Cascaded Refinement: Kimi-K2.5 performs cascaded OCR verification, structural calibration, and global consistency checking after fusion, preserving intermediate annotations and revision records for traceability.The pipeline separates high-recall candidate generation from higher-precision image-grounded correction.
- Data Selection: Approximately 100,000 validated in-domain pages are retained at roughly a 1% page-level retention rate for curated training data.The retained data covers major financial workflow scenarios and is later mixed with approximately 100,000 pages of out-of-domain public data.
4 Evaluation
FinixDocBench targets the deployment gap left by clean, conventional benchmarks by evaluating financial documents across digital, camera-captured, ultra-large, and internal-workflow settings. FinixDoc-VL performs strongly across these challenging scenarios, with especially large gains in low-quality and oversized-page processing.
- Benchmark Design: FinixDocBench covers digitally native, camera-captured, ultra-large-page, and internal financial-workflow scenarios in approximately 5,000 pages.Its four tracks are FinixDigital, FinixPhoto, FinixHuge, and FinixInner.
- General Document Parsing: FinixDoc-VL achieves ODB = 92.65 on OmniDocBench, while specialized document models remain ahead on this clean benchmark.It also achieves TEDS-Stable = 92.76 and Eorder = 0.055.
- Financial Digital Documents: 93.19 SFDB on FinixDigital is the best result, supported by Etext = 0.039 and TEDStable = 92.07.The result reflects strong recognition of dense financial terminology and structural fidelity in complex tables.
- Camera-Captured Documents: 67.03 SFDB and 77.69 TEDS-Stable are the best FinixPhoto results, with a 12.75-point SFDB improvement over Qwen3-VL-4B.The result indicates that financial-domain visual adaptation and business-aligned reinforcement learning improve parsing under mobile-capture degradation.
- Ultra-Large Pages: 0.92 Success Rate is achieved on FinixHuge, compared with 0.68 for Qwen3-VL-235B-A22B-Instruct and 0.34 for GLM-OCR.FinixDoc also records SFDB = 68.23, Etext = 0.357, and Eorder = 0.167.
- Evaluation Caveat: FinixDoc’s table-level FinixHuge scores do not lead because evaluator timeouts assign timed-out long predictions a TEDS score of 0.Approximately 25% of its table cases exceed the six-minute timeout, while baseline scores use only successfully processed pages.
- Overall Finding: The FinixHuge findings support treating ultra-large document parsing as a system-level problem rather than one solved by model scaling alone.Dynamic routing, structure-aware segmentation, and reconstruction improve processability when single-pass inference is unreliable.
- Internal Workflows: FinixDoc achieves its strongest FinixInner results on Claim Medical Records (85.42), Expense Statements (80.79), and Medical Examination Reports (78.38).These subsets combine dense terminology, complex tables, difficult reading orders, and mobile-capture degradation.
5 Related Work
Prior document parsing systems range from modular OCR and layout pipelines to unified general and specialized vision-language models. Their strengths remain divided: specialized models excel on clean structured documents, while general VLMs offer broader robustness, leaving low-quality, large-scale, and risk-sensitive financial settings insufficiently addressed.
- Traditional Pipelines: Traditional systems decompose parsing into OCR, layout analysis, table parsing, extraction, and rule-based post-processing.This architecture is mature and suits stable templates, regular layouts, and clearly defined tasks.
- Traditional Pipelines: Modular pipelines incur integration and maintenance costs, while upstream recognition or layout errors can propagate into later stages.Their reliance on task-specific components constrains end-to-end performance ceilings.
- General VLMs: General VLMs jointly model visual appearance, text, spatial layout, and cross-modal semantics within a unified framework.They can generate structured outputs directly from document images and support broad document-understanding tasks.
- General VLMs: General VLMs often show greater robustness under blur, shadows, perspective distortion, and other distribution shifts than document-specific models.However, they may remain weaker on dense tables, fine-grained recognition, long structured outputs, and visually unsupported content.
- Specialized Models: Specialized document models generally perform strongly on clean scans, digital documents, table parsing, reading order, and structural consistency.Their gains are concentrated on clean and regular distributions, where camera-captured documents remain underrepresented.
- Open Gap: Existing work leaves capability boundaries insufficiently explored under low-quality, large-scale, and risk-sensitive financial conditions.This gap motivates both the capability matrix and FinixDoc’s system design.
6 Conclusion
FinixDoc addresses the gap between strong conventional-benchmark performance and practical financial deployment by combining a capability matrix, an agentic parsing system, domain-adapted training, a human-in-the-loop data pipeline, and realistic benchmarks. The paper concludes that progress should prioritize robust, verifiable, and deployment-oriented capability beyond clean benchmarks.
- Conclusion: FinixDoc studies why document parsers fall short in financial deployment when documents are low-quality, structurally complex, or extremely large.It introduces a capability matrix, an end-to-end agentic system, domain-adapted training, a human-in-the-loop Data Factory, and realistic benchmarks.
- Conclusion: Evaluation should prioritize stability, robustness, and usability on low-quality, high-complexity, and large-scale documents rather than conventional benchmarks alone.The paper frames this as an industrial perspective on measuring document parsing progress.
- Future Work: Future work targets uncertainty-aware generation and refusal, unified multimodal modeling for large-scale pages, and broader FinixDocBench coverage.The stated motivation is that incorrect extraction can be worse than missing a field in financial scenarios.
- Conclusion: The capability matrix, FinixDoc system, Data Factory, and benchmarks provide a foundation for advancing financial document parsing under real deployment constraints.The broader goal is to shift research toward robust, verifiable, and deployment-oriented capabilities.
A Implementation Details of the Split-then-Merge Strategy
The split-then-merge strategy converts ultra-large-page parsing from a monolithic inference step into adaptive routing, local-region parsing, and page-level reconstruction. It preserves direct parsing for feasible pages while using structure-aware partitioning and direction-aware merging for difficult oversized inputs.
- Overview: The strategy supplements FinixDoc-VL by activating split-then-merge processing when page scale or layout complexity makes single-pass inference unreliable.Standard pages retain the native direct-parsing path.
- Pipeline Overview: Figure 8 organizes the pipeline into Dynamic Routing, Structure-Aware Segmentation, and Reconstruction.Pages are either sent directly to the VLM or routed through splitting and merging for ultra-large-page handling.
- Dynamic Routing: Routing uses pixel scale, aspect ratio, information density, estimated token load, and structural complexity to select direct parsing or splitting.Overflow triggers splitting, feasible regular pages use direct parsing, and complex pages use a complexity fallback.
- Structure-Aware Segmentation: Structure-aware segmentation selects projection-aligned cuts near low-density regions instead of uniform equal-size partitions.A sliding-window fallback handles weak projection signals, and fixed overlap preserves boundary context.
- Structure-Aware Segmentation: The segmented regions preserve reading continuity and structural integrity while supporting independent and parallel FinixDoc-VL parsing.This design is intended to improve suitability for deployment on oversized pages.
- Reconstruction: Reconstruction merges local outputs vertically for ultra-long pages and horizontally for ultra-wide layouts such as dense tables.Overlap-aware deduplication, row-aware concatenation, and line-level alignment support page-level integration.
- Reconstruction: A final integrity check suppresses duplicates, repairs minor discontinuities, and enforces the target Markdown representation.The stage is intended to preserve completeness and reading-order consistency on FinixHuge-style pages.
- Algorithm: Algorithm 1 formalizes the procedure using an input image I, base parser M, token budget τ, and structured Markdown output Y.It extracts features, estimates token load and complexity, routes the page, splits when needed, and reconstructs the result.
A.5 Discussion
FinixDoc’s split-then-merge strategy is a preprocessing-layer extension that makes ultra-large-page parsing more tractable without changing FinixDoc-VL’s core architecture. It preserves direct parsing for standard pages while reducing context and memory pressure for oversized pages.
- A.5 Discussion: The split-then-merge strategy leaves FinixDoc-VL’s core model architecture unchanged.It operates as a lightweight engineering mechanism within the Document Preprocessing Tool Layer.
- A.5 Discussion: For standard pages, adaptive routing preserves the direct-parsing path with minimal overhead.
- A.5 Discussion: For oversized pages, splitting reduces context-length and memory pressure before parsing.
- A.5 Discussion: Structure-aware splitting and direction-aware merging improve usability and stability on ultra-large inputs.The reconstruction stage restores page-level continuity as much as possible.
B Financial Homoglyph Vocabulary and Auxiliary Lexicons
The financial homoglyph vocabulary supports hard-negative construction by collecting visually confusable characters from real financial documents and filtering candidates with auxiliary lexicons.
- B Financial Homoglyph Vocabulary and Auxiliary Lexicons: The vocabulary stores visually confusable characters frequently observed in real financial documents.
- B Financial Homoglyph Vocabulary and Auxiliary Lexicons: Multi-view similarity filtering constructs the vocabulary using stroke-order and stroke-count lexicons.
B.1 Financial Homoglyph Vocabulary
The financial homoglyph vocabulary broadens visual-confusion modeling for degraded financial documents and generates realistic hard negatives for contrastive learning. It covers diverse character types and uses candidate retrieval, similarity filtering, and auxiliary stroke representations.
- B.1 Financial Homoglyph Vocabulary: Homoglyph includes strict glyph-level lookalikes and near-homoglyph confusions that are difficult to distinguish in low-quality images.Small deviations in digits, field values, names, and medical or reimbursement terminology may create disproportionate downstream business risk.
- B.1 Financial Homoglyph Vocabulary: The vocabulary covers numerals, English letters, Roman numerals, symbol-like characters, and frequent Chinese characters.It is tailored to financial, insurance, medical, and reimbursement documents.
- B.1 Financial Homoglyph Vocabulary: For each character c, candidate sets contain visually similar or perceptually confusable alternatives used for hard-negative construction.Characters are selectively substituted in ground-truth annotations to create lexically close negatives with realistic visual ambiguity.
- B.1 Financial Homoglyph Vocabulary: The vocabulary is implemented as a key–value mapping from target characters to candidate homoglyph sets.Representative entries include confusions such as “0” with “O”, “8”, “6”, “9”, and “D”.
- B.1 Financial Homoglyph Vocabulary: Replacement sampling follows predefined rules so contrastive learning distinguishes visually confusable alternatives relevant to financial business risk.
- B.1 Financial Homoglyph Vocabulary: Stroke-order data represents each character with aligned geometric-type and descriptive-name sequences for candidate retrieval.The stroke-type sequence uses compact codes such as S, H, P, D, and Z.
B.3 Stroke-Count Lexicon
The stroke-count lexicon supplies an auxiliary writing-complexity cue for homoglyph candidate filtering. Similar stroke counts can indicate confusions that become likely under severe image degradation.
- B.3 Stroke-Count Lexicon: The stroke-count lexicon directly supports computation of stroke-count similarity Scount(ci, cj).It provides a lightweight auxiliary cue for writing complexity.
- B.3 Stroke-Count Lexicon: Characters with identical or highly similar stroke counts, such as “清” and “情”, are particularly susceptible to confusion under camera blur or low optical resolution.
- B.3 Stroke-Count Lexicon: The lexicon maps each candidate character directly to its integer stroke count.Table 11 presents representative characters grouped by stroke count.
C Qualitative Comparison Results on FinixDigital and FinixPhoto
The qualitative evaluation compares FinixDoc with specialized and general-purpose baselines across digitally native and camera-captured financial documents. Its four-way layouts are designed to assess structural fidelity, content accuracy, and robustness under matched inputs.
- FinixDigital: FinixDigital comparisons use MinerU 2.5 and Qwen3-VL-235B-A22B-Instruct on dense tables, deep heading hierarchies, and superscript footnotes.
- FinixPhoto: FinixPhoto comparisons use Youtu-Parsing and Kimi-K2.5 on blur, perspective distortion, printing artifacts, and hallucination beyond visible content.
- Comparison Layout: Each example presents the original input alongside outputs from FinixDoc, the best specialized model, and the best general-purpose VLM.
C.1 FinixDigital: Comparison with MinerU 2.5 and Qwen3-VL-235B-A22B-Instruct
The FinixDigital comparisons examine rendered Markdown outputs on digitally native financial documents, emphasizing table structure, heading hierarchy, and superscript footnote preservation.
- Comparison Setup: FinixDigital compares FinixDoc with MinerU 2.5 and Qwen3-VL-235B-A22B-Instruct using side-by-side rendered Markdown outputs.
- Structural Fidelity: FinixDoc preserves the two-column disease-list table, heading hierarchy, and superscript footnote references in an insurance clause page.
- Structural Fidelity: FinixDoc reconstructs the numeric table grid, heading hierarchy, and consistent numbered-list formatting in an insurance product specification page.
- Structural Fidelity: FinixDoc accurately retains nested headings, table headers, and superscript footnote references on an insurance contract page.
C.2 FinixPhoto: Comparison with Youtu-Parsing and Kimi-K2.5
The FinixPhoto comparisons evaluate field-level parsing on camera-captured medical receipts and invoices, focusing on numerical accuracy, OCR robustness, and avoiding unsupported content.
- Comparison Setup: FinixPhoto uses a 2×2 grid with the original image and model overlays showing correct or incorrect parsing for selected fields.
- Field Accuracy: FinixDoc correctly parses all four highlighted fields on a Shanghai inpatient receipt, including the fee-detail table and three numeric values.
- Robustness: FinixDoc gets the invoice number, Chinese-capital total, and payee correct on a Chongqing receipt without hallucinating content beyond the visible front.
- Field Accuracy: FinixDoc correctly parses all four highlighted fields on a Hubei medical invoice, including the receipt code, social-credit code, date, and cashier code.