Source-linked AI summary
Docling Technical Report
Christoph Auer, Maksym Lysak, Ahmed Nassar, Michele Dolfi, Nikolaos Livathinos, Panos Vagenas, Cesar Berrospi Ramis, Matteo Omenetti, Fabian Lindlbauer, Kasper Dinkla, Lokesh Mishra, Yusik Kim, Shubham Gupta, Rafael Teixeira de Lima, Valery Weber, Lucas Morin, Ingmar Meijer, Viktor Kuropiatnyk, Peter W. J. Staar
TL;DR
PDF variability and weak standardization have long made machine-processable conversion challenging, while few open-source tools cover PDF conversion. Docling combines PDF processing with specialized layout and table models in a locally run, extensible package. It converts documents into structured JSON or Markdown and supports downstream search, retrieval, and classification workflows.
Problem
PDF variability, weak standardization, and print-oriented design make conversion to machine-processable formats challenging, while only a handful of open-source tools cover PDF conversion.
Source: §1 · p. 1Method
Docling combines PDF parsing and a customizable model pipeline with specialized layout-analysis and table-structure models.
Source: §3.2 · p. 3, Layout Analysis Model · p. 3, Table Structure Recognition · p. 3Results
Docling converts PDFs to JSON or Markdown, recovers page layout and table structures, extracts metadata, and optionally applies OCR.
Source: §1 · p. 2Takeaways & Limitations
Its structured output supports document search, passage retrieval, classification, and knowledge-extraction pipelines.
Source: §5 · p. 5EasyOCR transcription runs slowly on CPU, taking upwards of 30 seconds per page.
Source: Table Structure Recognition · p. 4
Abstract
from arXiv · showhide
This technical report introduces Docling, an easy to use, self-contained, MIT-licensed open-source package for PDF document conversion. It is powered by state-of-the-art specialized AI models for layout analysis (DocLayNet) and table structure recognition (TableFormer), and runs efficiently on commodity hardware in a small resource budget. The code interface allows for easy extensibility and addition of new features and models.
1 Introduction
PDF variability and weak standardization make conversion to machine-processable formats difficult, while open-source tools cover less than proprietary solutions. Docling addresses this gap with a locally run, extensible open-source converter and a range of PDF-processing capabilities.
1 Introduction
- PDF variability, weak standardization, and print-oriented design make machine-processable conversion difficult, while only a handful of open-source tools cover PDF conversion.The report notes that proprietary and cloud solutions dominate the available document-understanding tools.Source: §1 · p. 1
- Docling is a self-contained, locally run Python library built on specialized layout-analysis and table-structure models, with an architecture designed for adding features and models.The report identifies the models and datasets as work the team had developed and presented previously.Source: §1 · p. 1
- Docling converts PDFs to JSON or Markdown, recovers layout and tables, extracts metadata, and optionally applies OCR.It can also be configured for batch or interactive operation and use accelerators such as GPUs or MPS.Source: §1 · p. 2
2 Getting Started
Docling is installed as a package, with documentation and examples in its repository. Its interface converts individual PDFs or batches to JSON or Markdown, with options for configuring pipeline features and runtime.
2 Getting Started
- Install the docling package from PyPI; model assets download to a local Hugging Face cache on first use unless preinstalled.Documentation and examples are available in the GitHub repository.Source: §2 · p. 2
- The interface converts PDFs from files, URLs, or binary streams into JSON or Markdown, with separate methods for single documents and batches.The passage illustrates a conversion from a PDF URL and rendering the result as Markdown.Source: §2 · p. 2
- Runtime options include enabling or disabling OCR and table recognition, setting input-size limits, and choosing a CPU-thread budget.A Dockerfile is also provided for installation and execution in a container.Source: §2 · p. 2
3 Processing pipeline
Docling parses each PDF, applies page-level models, and assembles the predictions into a document that can be serialized as JSON or Markdown. Its backend choices, specialized models, and customizable pipeline support conversion from parsing through output assembly.
3 Processing pipeline
- The default pipeline parses PDF text and page images, applies models page by page, then aggregates predictions and post-processes them into a typed document for JSON or Markdown output.Post-processing augments metadata, detects language, and infers reading order.Source: §3 · p. 2
3.1 PDF backends
- The PDF backend must retrieve text tokens and page coordinates and render page images; available parsers presented obstacles including restrictive licensing, poor speed, or quality issues.The report offers multiple backends, including a qpdf-based custom parser in docling-parse and a pypdfium alternative.Source: §3.1 · pp. 2–3
3.2 AI models
- Docling initially releases a layout-analysis object detector and TableFormer for table-structure recognition, with pretrained weights and separate inference code.Both models also power the open-access deepsearch-experience service.Source: §3.2 · p. 3
Layout Analysis Model
- The layout model predicts page-element bounding boxes and classes from page images, using an architecture derived from RT-DETR and retrained on DocLayNet and proprietary datasets.The pipeline feeds it 72-dpi page images, then post-processes overlapping proposals and intersects them with PDF text tokens to form document units.Source: Layout Analysis Model · p. 3
Table Structure Recognition
- TableFormer predicts table row and column structure and identifies header and body cells, handling features such as empty cells, spans, and hierarchical headings.It receives detected table crops and included text cells; post-processing matches predictions back to PDF cells to avoid retranscribing table text.Source: Table Structure Recognition · pp. 3–4
- OCR is optional for scanned PDFs and bitmap content, but EasyOCR runs slowly on CPU, at upwards of 30 seconds per page.The report seeks additional OCR backends and speed improvements.Source: Table Structure Recognition · p. 4
3.3 Assembly
- Assembly aggregates page predictions into a document object, then post-processing augments features, corrects reading order, matches figures to captions, and labels metadata.The resulting object can be serialized as JSON or transformed into Markdown.Source: §3.3 · p. 4
3.4 Extensibility
- The model pipeline can be customized by subclassing or cloning the default, allowing models to be added or replaced and configuration parameters introduced.Implementations accept an iterator of page objects and return page objects augmented with predicted features.Source: §3.4 · p. 4
4 Performance
The section establishes Docling’s runtime and resource profile using a 225-page test set, two PDF backends, two systems, and two thread budgets. It also notes trade-offs and ongoing work around low-resource use and GPU acceleration.
4 Performance
- Tests used 225 pages from three arXiv papers and two IBM Redbooks, across two systems, both PDF backends, and 4- or 16-thread budgets.OCR was disabled; Table 1 reports time-to-solution, throughput, and peak memory.Source: §4 · p. 4, Table 1 · p. 5
- The pypdfium backend is faster and more memory efficient than docling-parse, but produces lower-quality results, especially for table structure recovery.It is suggested for very low-resource environments.Source: §4 · p. 4
- GPU acceleration for the AI models remains work-in-progress and largely untested, though it may work when CUDA is available to the supporting runtimes.The report says it will provide updates in a future version.Source: §4 · pp. 4–5
5 Applications
Docling’s structured output is presented as a base for document-focused search, retrieval, classification, and knowledge extraction. The report also describes uses in RAG, dataset building, and knowledge-base construction.
5 Applications
- Docling output can support enterprise document search, passage retrieval, classification, and knowledge-extraction pipelines that treat tables, figures, sections, and references differently.The report attributes these applications to high-quality, richly structured conversion.Source: §5 · p. 5
- The open-source quackling package uses Docling output for document-native vector embedding and chunking in RAG workflows, including with LlamaIndex.The report says it plugs into LLM frameworks such as LlamaIndex.Source: §5 · p. 5
- Docling is presented as useful for building document-derived datasets and for automated knowledge-base construction, and is integrated into IBM’s open data prep kit.The data prep kit implements scalable transforms for large-scale multimodal training datasets.Source: §5 · p. 5
6 Future work and contributions
The report’s roadmap includes extending Docling’s model library and pipelines, improving GPU acceleration and its native PDF backend, and reviewing community contributions under the MIT license.
6 Future work and contributions
- Planned model additions include figure classification, equation recognition, and code recognition to improve conversion for specific content and augment extracted metadata.The report describes Docling as designed for easy extension of its model library and pipelines.Source: §6 · p. 5
- Further GPU-acceleration testing and optimization, along with improvements to the Docling-native PDF backend, are on the roadmap.Source: §6 · p. 5
- The authors invite feature and model contributions for review, with the codebase open for use and contribution under the MIT license.Source: §6 · p. 5
Appendix
The appendix figures illustrate Docling’s rendered output and document-processing behavior, alongside examples of layout-model evaluation and dataset scaling. The captions describe the comparisons and transformations shown without supplying benchmark values.
Appendix
- Figure 2 contrasts the DocLayNet title page PDF with rendered Markdown, where recognized metadata appears first and figure-internal text is dropped.The caption is retained and linked to the figure in the JSON representation.Source: Figure 2 · p. 7
- Table 2 reports DocLayNet test-set mAP@0.5-0.95 for object detectors including Mask R-CNN, Faster R-CNN, and YOLOv5x6 [13].The caption describes model backbones and COCO 2017 pre-trained initialization but gives no cell values.Source: Table 2 · p. 8
- Figure 5 shows Mask R-CNN with a ResNet50 backbone reaching a learning-curve plateau near 80% of DocLayNet training data.The caption says adding similar data beyond that point will not yield significantly better predictions.Source: Figure 5 · p. 8
- Figure 3 illustrates reading-order output: headers and footers are suppressed, tables are inserted inline, and a cross-column paragraph can be interrupted by a table.Source: Figure 3 · p. 8
- Figure 4 contrasts the original table, Markdown, and JSON representations: Markdown repeats spanning headers, while JSON records span information in cell fields.Source: Figure 4 · p. 9