Source-linked AI summary
Docling: An Efficient Open-Source Toolkit for AI-driven Document Conversion
Nikolaos Livathinos, Christoph Auer, Maksym Lysak, Ahmed Nassar, Michele Dolfi, Panos Vagenas, Cesar Berrospi Ramis, Matteo Omenetti, Kasper Dinkla, Yusik Kim, Shubham Gupta, Rafael Teixeira de Lima, Valery Weber, Lucas Morin, Ingmar Meijer, Viktor Kuropiatnyk, Peter W. J. Staar
TL;DR
Document conversion must handle highly variable formats while preserving structure, and existing solutions often involve licensing, deployment, cost, or fidelity trade-offs. Docling addresses this with a local, open-source toolkit combining specialized layout and table models with a unified document representation. It provides broad format support and competitive conversion speed across CPU and GPU settings, while its roadmap identifies further models and transparent quality evaluation as future work.
Problem
Document conversion is difficult because formats vary widely, standardization is weak, and printing-oriented files often discard structure and metadata.
Method
Docling combines local document-conversion pipelines, a unified DoclingDocument model, and specialized layout-analysis, table-recognition, and OCR components.
Results
Docling supports multiple document formats and achieves 3.1 sec/page on x86 CPU, 1.27 sec/page on M3 Max, and 0.49 sec/page on an Nvidia L4 GPU.
Takeaways & Limitations
Docling offers a local, permissively licensed conversion system whose structured outputs can support downstream applications and generate ground-truth data for model training and fine-tuning.
Takeaways & Limitations
The authors identify future needs for additional content-specific models and an open-source quality-evaluation framework across Docling’s tasks.
Abstract
from arXiv · showhide
We introduce Docling, an easy-to-use, self-contained, MIT-licensed, open-source toolkit for document conversion, that can parse several types of popular document formats into a unified, richly structured representation. It is powered by state-of-the-art specialized AI models for layout analysis (DocLayNet) and table structure recognition (TableFormer), and runs efficiently on commodity hardware in a small resource budget. Docling is released as a Python package and can be used as a Python API or as a CLI tool. Docling's modular architecture and efficient document representation make it easy to implement extensions, new features, models, and customizations. Docling has been already integrated in other popular open-source frameworks (e.g., LangChain, LlamaIndex, spaCy), making it a natural fit for the processing of documents and the development of high-end applications. The open-source community has fully engaged in using, promoting, and developing for Docling, which gathered 10k stars on GitHub in less than a month and was reported as the No. 1 trending repository in GitHub worldwide in November 2024.
1 Introduction
Docling is an open-source, locally runnable document-conversion toolkit built to address highly variable formats and limited structural metadata. It combines specialized document-understanding models with a unified representation and broad format support.
- Contribution: Docling is a self-contained, MIT-licensed Python toolkit that runs locally on commodity hardware and is designed for extensibility.It builds on specialized AI models and supports new features and models through its code architecture.
- Models: It releases open-source layout-analysis and table-structure-recognition models alongside the conversion toolkit.These models provide the specialized AI foundation for Docling’s document understanding capabilities.
- Capabilities: It parses PDF, image, Office, and HTML inputs and exports Markdown, JSON, and HTML.The toolkit applies layout analysis, OCR, reading-order detection, figure extraction, and table-structure recognition.
- Representation: Docling uses a unified DoclingDocument model for rich document representation and downstream processing.The model supports inspection, export, and chunking through APIs and integrates with frameworks such as LangChain and LlamaIndex.
2 State of the Art
Document-conversion systems differ in licensing, deployment, fidelity, speed, and resource requirements. Docling occupies the local, open-source category while emphasizing specialized models, faithful conversion, permissive licensing, and resource efficiency.
- Comparison dimensions: Document-conversion solutions span open and closed source, restrictive and permissive licenses, web APIs and local deployment, quality, speed, and compute requirements.These dimensions frame comparisons among available conversion tools.
- Vision-language models: Vision-language models process page images directly but face hallucination concerns when faithful transcription is required.The passage identifies false information absent from the source document as a critical issue.
- On-premises tools: On-premises document-conversion tools commonly combine specialized OCR, layout-analysis, and table-recognition models.Docling belongs to this category and recovers structure and features while taking text from PDFs or OCR.
- Docling: Docling distinguishes itself with a permissive MIT license that avoids licensing fees and restrictive-license requirements.This supports organizational integration without adopting licenses such as GPL.
- Docling: Docling is presented as accurate, resource-efficient, fast, cost-effective, transparent, and flexible for document conversion.Its resource-efficient models are described as suitable for integration with standard frameworks.
3 Design and Architecture
Docling organizes document conversion around parser backends, processing pipelines, and the DoclingDocument data model. This architecture unifies heterogeneous formats while supporting inspection, export, chunking, customization, and format-specific processing.
- Architecture: Docling’s architecture centers on pipelines, parser backends, and the DoclingDocument model as the common representation.Pipelines and backends construct and enrich documents, while the model supports downstream processing such as RAG.
- DoclingDocument: DoclingDocument represents text, tables, pictures, captions, lists, hierarchy, layout, and provenance across source formats.It provides a unified representation regardless of the input document format.
- DoclingDocument: DoclingDocument APIs support incremental construction, reading-order traversal, inspection, lossless JSON serialization, and lossy Markdown or HTML export.Markdown and HTML cannot retain all available metadata, unlike JSON.
- Downstream processing: Chunkers stream document portions as strings with metadata and can connect downstream applications to LangChain or LlamaIndex.The chunker hierarchy includes a base type and specialized subclasses for flexible downstream strategies.
- Format handling: Low-level formats require specialized recovery of semantics, whereas markup-based formats preserve structure and are comparatively inexpensive to parse.Docling routes these format classes through fitting parser backends and pipelines.
- PDF backends: A custom qpdf-based PDF parser was developed to address limitations involving licensing, speed, and unrecoverable text or table quality issues.The parser is provided as the default PDF backend in the separate docling-parse package.
- Pipelines: Docling provides Standard-PdfPipeline for PDFs and images and SimplePipeline for markup-based formats.Pipelines orchestrate parser outputs and chains of models, and users can customize them by subclassing or cloning defaults.
4 PDF Conversion Pipeline
Docling’s PDF pipeline combines programmatic PDF text and rendered page images with specialized models for layout, tables, OCR, and post-processing. The outputs are aggregated into a structured DoclingDocument representation.
- Pipeline overview: The PDF pipeline retrieves text tokens and coordinates, renders each page, applies page-level AI models, and aggregates results into DoclingDocument.Image inputs are wrapped in PDF containers and processed as scanned PDFs.
- Layout Analysis Model: Docling releases a layout-analysis model that detects page-element bounding boxes and classes using an RT-DETR-derived architecture trained on DocLayNet.Predictions are post-processed and intersected with PDF text tokens to form meaningful document units.
- Table Structure Recognition: TableFormer is a vision transformer that recovers table rows, columns, headers, body cells, spans, and hierarchical structure from table images and text cells.Its predictions are matched back to PDF cells, avoiding expensive image re-transcription and making the model language agnostic.
- OCR: OCR converts scanned PDFs and bitmap images, with EasyOCR and Tesseract provided as integrations.EasyOCR offers reasonable transcription quality but is the pipeline’s largest compute expense on CPU.
- Post-processing: Docling assembles page-level predictions into DoclingDocument and post-processes them to correct reading order and match figures with captions.This is the final pipeline stage after model inference on individual pages.
5 Performance
Docling’s performance evaluation benchmarks local PDF conversion across a varied dataset, three system configurations, and comparable open-source tools. Runtime depends strongly on page complexity and optional AI processing, while GPU acceleration benefits models unevenly.
- Benchmark setup: The benchmark covers 89 PDF files, 4008 pages, 56,246 text items, 1,842 tables, and 4,676 pictures.The dataset combines DocLayNet with additional CCpdf samples to increase variety.
- Benchmark setup: Experiments use three configurations: M3 Max SoC, Nvidia L4 GPU, and x86 CPU.The configurations derive from an M3 Max MacBook Pro and an AWS EC2 system tested with GPU and CPU execution.
- Runtime characteristics: Document length does not determine conversion time linearly because tables and bitmap elements dynamically trigger table recognition and OCR.Pages with different content complexity therefore require different processing paths.
- Runtime characteristics: 0.79 seconds per page is the median on x86 CPU, compared with 0.32 seconds on M3 Max and 114 milliseconds on Nvidia L4 GPU.The reported 5th-to-95th percentile ranges are 0.6–16.3 seconds on x86, 0.26–6.48 seconds on M3 Max, and 57–2081 milliseconds on L4.
- Runtime characteristics: Disabling both OCR and table structure recognition saves around 75% of runtime across all system configurations.OCR alone saves 60% on x86 and M3 Max and 50% on L4; table recognition saves 16%, 16%, and 24%, respectively.
- Runtime profiling: 481 ms per page on L4, 3.1 s on x86, and 1.26 s on M3 Max are the average benchmark processing times.The PDF backend contributes substantially less time than the AI models, averaging 81 ms on x86 and 44 ms on M3 Max.
- Runtime profiling: L4 GPU acceleration yields 8x, 14x, and 4.3x speedups for OCR, layout, and table structure versus x86 CPU.Compared with the M3 Max CPU, the corresponding speedups are 3x, 6x, and 1.7x, showing unequal benefits across models.
- Tool comparison: Docling leads CPU conversion with 3.1 sec/page on x86 and 1.27 sec/page on M3 Max, while MinerU leads with CUDA acceleration.The comparison includes Marker, MinerU, and Unstructured under locally runnable configurations.
6 Applications
Docling’s structured extraction supports generative-AI workflows, training-data preparation, fine-tuning, and information extraction. Its integrations offer both lossy text exports and lossless structured representations for downstream processing.
- Applications: Docling’s document extraction supports RAG, foundation-model training and fine-tuning, and information extraction.These applications use the structured content recovered from diverse document formats.
- Retrieval-augmented generation: RAG users can connect Docling extensions to LlamaIndex and choose Markdown export or lossless JSON serialization.Markdown supports text-based chunking, while JSON preserves richer document structure for a swappable Docling chunker.
- Training and fine-tuning: Docling-enabled pipelines can generate ground-truth data from documents for foundation-model training and fine-tuning.The paper links this use to incorporating domain-specific knowledge into those processes.
- Information extraction: Docling supports information extraction by mapping formats into DoclingDocument and applying table understanding to semi-structured content.The unified representation provides structured inputs for downstream extraction tasks.
7 Ecosystem
Docling is presented as a locally runnable, broadly integrated document-conversion package with support across formats and downstream AI workflows. Its ecosystem spans retrieval, dataset creation, fine-tuning, information extraction, and agentic applications.
- Package ecosystem: Docling supports PDF, MS Office formats, images, HTML, and local execution on commodity hardware under a permissive MIT license.The paper also highlights richly structured output and library intuitiveness.
- Integrations: Integrations connect Docling to LangChain, LlamaIndex, IBM data-prep-kit, and Bee for retrieval, dataset creation, and agentic workloads.The ecosystem includes integrations contributed by both the Docling team and the broader community.
- Performance context: Figure 4 separates PDF-backend and AI-model contributions to seconds per page, while Figure 5 compares conversion times across tools and configurations.The figures use lower-is-better runtime comparisons and include OCR and table-structure processing.
- Enterprise deployment: Docling is officially maintained as a system package in the RHEL AI distribution for enterprise Granite-model workflows.This places the package within an enterprise distribution for developing, testing, and running Granite models.
8 Future Work and Contributions
Docling’s modular architecture supports extensions to its model library and pipelines, while its MIT-licensed codebase invites community improvements and contributions. Future work includes additional content-specific models and an open-source quality evaluation framework.
- Extensibility: Docling’s modular architecture supports extending its model library and conversion pipelines.The paper highlights this architecture as the basis for adding new features and models.
- Planned models: Planned models target figure classification, equation recognition, and code recognition to improve conversion quality for specific content.These models are also intended to augment extracted document metadata.
- Evaluation: The authors plan an open-source quality evaluation framework covering layout analysis, table structure recognition, reading order, and text transcription.The framework is intended to support transparent comparisons on publicly available benchmarks.
- Community contributions: Docling’s MIT license and repository roadmap invite users to propose improvements and contribute to the codebase.The codebase is open for use under the MIT license, with the roadmap outlined in the repository discussions.