PaperlayerView exact revision on arXiv

Source-linked AI summary

MultiDocFusion: Hierarchical and Multimodal Chunking Pipeline for Enhanced RAG on Long Industrial Documents

Joongmin Shin, Chanjun Park, Jeongbae Park, Jaehyung Seo, Heuiseok Lim

arXiv:2604.12352v1cs.AIcs.CL

TL;DR

Long-document RAG is limited by chunking methods that ignore visual layout and explicit section hierarchy, causing context fragmentation in complex documents. MultiDocFusion combines document parsing, OCR, DSHP-LLM hierarchy reconstruction, and DFS-based grouping to create structure-aware chunks. Across diverse benchmarks, it improves retrieval precision and ANLS QA scores over baselines, while the authors identify limited visual grounding and unevaluated graph retrieval as boundaries of the work.

  • Problem

    Text-only chunking and vision-based parsing do not adequately preserve visual layout, semantic hierarchy, and context in complex long documents.

  • Method

    MultiDocFusion combines vision-based document parsing, OCR, DSHP-LLM hierarchy reconstruction, and DFS-based grouping into a multimodal chunking pipeline.

  • Results

    MultiDocFusion consistently outperforms baseline chunking methods in retrieval and QA across four multi-page VQA benchmarks and diverse domains and layouts.

  • Takeaways & Limitations

    Hierarchy-aware, visually grounded chunking is supported as a design principle for RAG on long, complex, and scanned industrial documents.

  • Takeaways & Limitations

    DSHP-LLM underuses fine-grained visual cues because its training data lacks detailed layout signals and its inputs primarily contain OCR text with coarse bounding boxes.

Abstract

from arXiv · show

RAG-based QA has emerged as a powerful method for processing long industrial documents. However, conventional text chunking approaches often neglect complex and long industrial document structures, causing information loss and reduced answer quality. To address this, we introduce MultiDocFusion, a multimodal chunking pipeline that integrates: (i) detection of document regions using vision-based document parsing, (ii) text extraction from these regions via OCR, (iii) reconstruction of document structure into a hierarchical tree using large language model (LLM)-based document section hierarchical parsing (DSHP-LLM), and (iv) construction of hierarchical chunks through DFS-based grouping. Extensive experiments across industrial benchmarks demonstrate that MultiDocFusion improves retrieval precision by 8-15% and ANLS QA scores by 2-3% compared to baselines, emphasizing the critical role of explicitly leveraging document hierarchy for multimodal document-based QA. These significant performance gains underscore the necessity of structure-aware chunking in enhancing the fidelity of RAG-based QA systems.

1 Introduction

Long-document RAG suffers when text-centric chunking ignores visual layout and hierarchical section relationships. MultiDocFusion addresses this with multimodal structure reconstruction and reports consistent retrieval and QA improvements across diverse documents.

  • Problem: Text-centric chunking fails to preserve visual and hierarchical information in long industrial and academic documents.Tables, figures, section headers, scanned layouts, and OCR artifacts can fragment semantic continuity and degrade retrieval and QA.
  • Approach: MultiDocFusion integrates vision-based document parsing, OCR, DSHP-LLM hierarchical parsing, and DFS-based chunk assembly.The pipeline combines visual layout with an explicit document hierarchy to produce structurally faithful chunks.
  • Results: The experiments consistently show improvements in retrieval precision and answer accuracy across diverse document types.The authors highlight explicit hierarchical modeling as essential for robust, context-aware question answering.
  • Approach: DSHP-LLM reconstructs hierarchical section structures from complex documents to preserve context for downstream retrieval and QA.The model is introduced as an instruction-tuned component for robust hierarchical reconstruction.
  • Results: 8–15% retrieval precision and 2–3% ANLS QA scores are reported as improvements across diverse industrial and academic document settings.The evaluation includes financial reports, technical documents, scanned images, and complex layouts.

2 Related Work

Prior chunking and document-parsing methods often miss explicit section hierarchy, visual layout, or full-document context. These limitations motivate structure-aware multimodal chunking for long, multi-page documents.

  • Chunking methods: Length and semantic chunking often fail to represent hierarchical relationships and visual elements such as tables and figures.These methods segment documents without comprehensively capturing structure and context.
  • Chunking methods: LLM-based chunking methods can still suffer fragmentation when they lack explicit document-hierarchy modeling.The related work identifies missing hierarchy as a continuing limitation.
  • Document structure: StyleDFS struggles with scanned documents that lack text layers or contain irregular layouts.Font-size and style cues are unavailable or unreliable in these settings.
  • Multimodal methods: End-to-end multimodal models face limited context lengths when processing entire multi-page documents.This creates a need for methods that capture document structure and context more comprehensively.
  • Document parsing: Object-detection-centric document parsing identifies visual components but does not fully reconstruct semantic relationships between sections.The cited example contrasts sections such as 1.2 and 1.2.1.

3 MultiDocFusion

MultiDocFusion converts document pages into annotated visual layouts, reconstructs a document hierarchy, and groups nodes into hierarchy-preserving chunks. Its chunks retain spatial and semantic relationships through Markdown headers and section-aware aggregation.

  • Pipeline overview: MultiDocFusion integrates DP, OCR, DSHP-LLM, and DFS-based Grouping to process diverse long-document formats.The pipeline targets PDFs, scanned images, complex layouts, and corpus-level multidocument RAG.
  • DP: DP detects document segments and records their types and bounding-box coordinates in page-level layout structures.Detected elements include titles, section headers, text blocks, tables, and figures.
  • OCR: OCR links recognized text to DP bounding boxes, producing annotated layouts that combine text, segment types, and spatial metadata.The annotated layout supplies structured inputs for subsequent hierarchy reconstruction.
  • DSHP-LLM: DSHP-LLM constructs a document hierarchical tree by ordering headers and attaching general nodes through parent–child relationships.The process links header structure with spatially sorted tables, figures, and text blocks.
  • DFS-based Grouping: DFS-based Grouping traverses the hierarchy and aggregates parent and child content into coherent chunks, splitting when length exceeds max_len.Markdown headers encode each node’s depth, and chunks can preserve complete sections or subsections.
  • Output: The final chunks preserve spatial and semantic relationships and outperform traditional text-only chunking in retrieval and QA scenarios.Hierarchical structure is explicitly marked with Markdown headers.

4 Experimental Settings

The experiments evaluate DSHP-LLM for hierarchical parsing and MultiDocFusion across document datasets, models, and retrieval/QA metrics.

  • Evaluation: The experimental section compares DSHP-LLM across fine-tuned LLMs and evaluates chunking methods for retrieval, QA, and robustness across pipeline components.The robustness analysis varies DP, OCR, and embedding model combinations.
  • Datasets: The evaluation uses DocHieNet and HRDH to train and test DSHP-LLM, plus DUDE, MPVQA, CUAD, and MOAMOB for multi-page RAG-based VQA.The datasets cover diverse domains, complex layouts, and multi-page documents.
  • Models: DP uses DETR and VGT, OCR uses Tesseract, EasyOCR, and TrOCR, and DSHP-LLM is instruction-tuned on Llama-3.2-3B, Qwen-2.5-3B, and Mistral-8B.The model predicts hierarchical parent-child relationships among document section headers.
  • Evaluation: DSHP-LLM performance is measured with accuracy, F1, and TEDS, while retrieval and VQA use Precision, Recall, nDCG, ANLS, ROUGE-L, and METEOR.These metrics cover hierarchy parsing, retrieval quality, and generated-answer quality.

5 Experimental Results

MultiDocFusion improves hierarchy parsing, retrieval, and QA across diverse multi-page document benchmarks, with gains remaining stable across DP, OCR, and embedding choices.

  • DSHP-LLM Performance: DSHP-LLM improves TEDS over general-purpose LLMs on DocHieNet and HRDH, with gains up to +52.25% for Mistral-8B on HRDH.GPT-4 without fine-tuning scores TEDS 0.6961 on DocHieNet and 0.3342 on HRDH; improvements vary by model and dataset.
  • Retrieval Performance: MultiDocFusion achieves Recall 0.2927, Precision 0.2001, and nDCG 0.2505 on DUDE, and Recall 0.2705, Precision 0.1759, and nDCG 0.2131 on MPVQA.It also leads Precision and nDCG on CUAD and all reported retrieval metrics on MOAMOB.
  • Retrieval Performance: MultiDocFusion improves retrieval by capturing hierarchical and semantic relationships that LLM-only chunking methods insufficiently incorporate.Compared with Structure-based Chunking on DUDE, it enhances Recall by 7.08% and Precision by 5.51%.
  • QA Performance: MultiDocFusion achieves ANLS 0.1615, ROUGE-L 0.1316, and METEOR 0.1850 on MPVQA, and ANLS 0.1859, ROUGE-L 0.1692, and METEOR 0.2285 on DUDE.It records the highest ANLS and ROUGE-L on CUAD and the highest ANLS and METEOR on MOAMOB.
  • Robustness: MultiDocFusion reaches average nDCG 0.5017 across DP models, up to +27.64% over Semantic chunking in VGT.The comparison covers DETR, DiT, and VGT environments.
  • Robustness: MultiDocFusion achieves average nDCG 0.4949 across OCR models and 0.5061 across embedding models, outperforming other chunking methods.Its best reported embedding result is 0.5213 in the BGE environment.

6 Conclusion

MultiDocFusion addresses context fragmentation in long industrial-document RAG by combining visual layout parsing, OCR, hierarchical reconstruction, and DFS chunk assembly. Across multi-page VQA benchmarks, it outperforms baseline chunking methods in retrieval and QA, supporting hierarchy-aware, visually grounded chunking.

  • Conclusion: MultiDocFusion parses layout regions, extracts OCR text, reconstructs section hierarchies with DSHP-LLM, and assembles hierarchical chunks using DFS.The pipeline preserves both spatial and semantic context.
  • Conclusion: Across four multi-page VQA benchmarks, MultiDocFusion consistently outperforms baseline chunking methods in retrieval and QA.DSHP-LLM also surpasses general-purpose LLMs on hierarchical parsing datasets.
  • Conclusion: The results support hierarchy-aware, visually grounded chunking as a first-class design principle for long, complex, and often scanned industrial documents.The conclusion links visual segmentation, an explicit document tree, and chunk boundaries.

Limitations

MultiDocFusion improves retrieval and QA but remains constrained by visual grounding, unvalidated graph retrieval, error propagation, and computational overhead.

  • Visual grounding: DSHP-LLM underuses fine-grained visual cues because its training data lacks detailed layout signals, limiting hierarchy reconstruction on visually complex pages.The paper identifies font, color, whitespace, alignment, ruling lines, and column structure as missing signals.
  • Unvalidated graph retrieval: Graph-structured retrieval is proposed but not systematically evaluated because the study focuses on standard RAG settings.The induced hierarchy could support node- and path-based retrieval for multi-hop and cross-page reasoning, but this remains future work.
  • Pipeline risks: Serial DP/OCR-to-QA processing is susceptible to error propagation, allowing early mistakes to cascade through hierarchy parsing, chunking, retrieval, and QA.The paper suggests studying end-to-end or hybrid VLM components as alternatives, while noting trade-offs in controllability and interpretability.
  • Computational overhead: Hierarchical chunking can increase index size, retrieval latency, and storage costs by duplicating parent context across child chunks.Budget-aware chunking, graph pruning, and node-level caching or deduplication are proposed mitigations.

Ethical Considerations

Applying MultiDocFusion to real documents requires privacy and security safeguards, alongside accountability measures for potentially incorrect or biased outputs.

  • Sensitive data: Documents processed by the pipeline may contain personal data, copyrighted material, or proprietary business content requiring careful collection, processing, and use.The paper calls for adherence to privacy regulations and data security standards.
  • Deployment safeguards: Incorrect or biased generated responses could mislead users, so practical deployments should include accountability guidelines and measures against misuse.The paper frames these safeguards as necessary despite the system’s goal of accurate information.

A.1 Dataset Details

The appendix describes datasets for hierarchical parsing and multi-page RAG QA across diverse document types, languages, domains, and structural layouts.

  • DSHP-LLM datasets: DocHieNet and HRDH provide hierarchical section annotations with parent-child relationships for DSHP-LLM training and evaluation.DocHieNet includes diverse reports, papers, and industrial documents, while HRDH emphasizes complex academic layouts.
  • DSHP-LLM datasets: DocHieNet contains 1,673 PDFs averaging 5.3 pages, including scanned and structurally diverse documents.Its annotations cover relationships among titles, paragraphs, tables, figures, and other elements.
  • DSHP-LLM datasets: HRDH contains approximately 1,500 arXiv PDFs averaging 7.1 pages and more than 30 complex layout types.Each line is labeled with its corresponding parent section.
  • Multi-page VQA datasets: MPVQA, CUAD, DUDE, and MOAMOB support practical RAG QA evaluation across industrial, legal, financial, and other document formats.The listed benchmarks vary substantially in size, from MOAMOB’s two documents to MPVQA’s approximately 17,000 documents.
  • Evaluation constraints: Official DUDE and MPVQA test sets hide ground-truth answers, preventing retrieval-metric computation on those test sets.The study therefore reports retrieval and QA results on validation splits and includes official test results separately for MPVQA.
  • Language coverage: The datasets cover English, Chinese, and Korean documents, with language coverage varying by benchmark.DocHieNet includes English and Chinese documents, MOAMOB includes Korean documents, and the remaining datasets are in English.

A.2 Model and Implementation Settings

The implementation evaluates MultiDocFusion across interchangeable parsing, OCR, hierarchy, embedding, and QA models, then assembles hierarchical chunks with DFS under length constraints.

  • Experimental setup: The experiments cross-apply multiple models for pipeline components to test robustness under varied realistic configurations.Components include document parsing, OCR, DSHP-LLM, embeddings, and QA generation.
  • Component models: Document parsing models detect page-level layout segments such as tables, figures, and text blocks, providing segment types and bounding boxes.The evaluated models include DETR, DiT, and VGT fine-tuned on DocLayNet.
  • Component models: OCR models extract text from detected segments, with accuracy varying by document quality, fonts, and language.The evaluated OCR systems include EasyOCR, Tesseract, and TrOCR.
  • Component models: DSHP-LLM models are instruction-tuned on JSON hierarchical structures from DocHieNet and HRDH, using LoRA and 4-bit quantization for parameter efficiency.The evaluated backbones include Llama-3.2-3B, Qwen-2.5-3B, Mistral-8B, and Qwen-2.5-7B.
  • Retrieval and QA: Retrieval compares BGE, E5, and BM25 embeddings with top-k retrieval fixed at 4, while QA uses Llama-3.2-3B, Mistral-8B, and Qwen-2.5-7B.Retrieved top-ranked chunks provide context for RAG-based answer generation.
  • Chunk statistics: MultiDocFusion produces 20,773 chunks averaging 521.65 tokens because identical parent headers duplicate context across children.Despite the larger chunk count, the paper reports superior retrieval performance.
  • Results: MultiDocFusion consistently outperforms other chunking methods across diverse datasets and remains robust across DP, OCR, and embedding choices.The appendix reports detailed comparisons across top-k values and component ablations.
  • Hierarchical chunking: The DFS chunker traverses the hierarchy tree, carries parent context into child sections, and splits accumulated text when it exceeds max_len.This preserves hierarchical structure while enforcing chunk-length constraints.

B.2 Results of Document Chunking Using Different Methods

MultiDocFusion preserves document context by combining visual layout information with hierarchical structure during chunking. The qualitative comparison shows that conventional methods often produce unclear boundaries, while MultiDocFusion creates hierarchically organized chunks.

  • Conventional text-based methods often lack clear segmentation criteria between chunks and fail to maintain contextual continuity.The comparison covers Length, Semantic, LumberChunker, and Perplexity chunking.
  • MultiDocFusion includes higher-level hierarchical nodes within each chunk to preserve contextual coherence.Its chunks are organized around reconstructed document structure rather than text segmentation alone.
  • The illustrated pipeline combines document parsing, OCR, DSHP-LLM hierarchy reconstruction, and DFS-based grouping for hierarchy-aware chunking.Document parsing detects layout regions such as titles, headers, text blocks, tables, and figures; OCR links recognized text to bounding boxes.
  • The qualitative comparison applies six approaches to three chunks from the nuclear-operations document shown in Figure 2.The approaches are Length, Semantic, LumberChunker, Perplexity, Structure-based, and MultiDocFusion.
  • The evaluation tables summarize retrieval, OCR, document-parsing, embedding, and generation performance across chunking methods.Generation results include ANLS, ROUGE-L, and METEOR across multiple datasets and top-k settings.
Loading 2604.12352v1…