Source-linked AI summary
HPD-Parsing: Hierarchical Parallel Document Parsing
Shu Wei, Jingjing Wu, Lingshu Zhang, Qunyi Xie, Hao Zou, Le Xiang, Xu Fan, Yangliu Xu, Manhui Lin, Xiaolong Ma, Cheng Cui, Tengyu Du, YY
TL;DR
Unified VLM document parsers face costly sequential decoding despite globally coordinated layouts and largely localized content. HPD-Parsing uses hierarchical parallel decoding with progressive multi-token prediction, reaching 4,752.1 TPS—3.06× the vanilla autoregressive baseline—while maintaining competitive parsing accuracy.
Problem
Unified VLM document parsers generate long structured outputs token by token, although page layout requires global coordination while region content is largely localized.
Method
HPD-Parsing replaces one full-page autoregressive trajectory with a globally coordinated layout branch, concurrent localized content branches, and progressive multi-token prediction.
Results
4,752.1 TPS versus 1,554.8 TPS for the autoregressive baseline, a 3.06× improvement, while maintaining competitive parsing accuracy.
Takeaways & Limitations
The results validate layout-coordinated parallel concurrency as an effective alternative to traditional single-trajectory autoregressive document parsing.
Abstract
from arXiv · showhide
Efficient teamwork typically combines global coordination with parallel execution, a principle not yet fully reflected in unified Vision-Language Model (VLM)-based document parsers. Existing unified parsers process an entire page jointly but generate its output through a single token-by-token autoregressive trajectory, creating a sequential bottleneck that grows with document length. Such full-page sequential generation overlooks a key property of document parsing: layout must be analyzed globally, whereas block content can be parsed in parallel. Based on this observation, we introduce HPD-Parsing, which replaces full-page autoregressive generation with a Hierarchical Parallel Decoding paradigm. A main layout branch organizes the overall document structure and dynamically assigns block-level content decoding to concurrent branches, while progressive multi-token prediction (P-MTP) further reduces the decoding steps within each branch. Experiments on public benchmarks show that HPD-Parsing achieves 4,752 tokens per second, delivering $2.62\times$ the throughput of the fastest existing document parsing model and $3.06\times$ that of the vanilla autoregressive baseline, while maintaining competitive parsing accuracy. These results establish hierarchical parallel decoding as an effective alternative to full-page autoregressive generation, opening a new direction for efficient unified document parsing.
1. Introduction
HPD-Parsing addresses the sequential bottleneck of unified VLM document parsers by separating globally coordinated layout analysis from localized, parallel content decoding. Its hierarchical decoding design combines concurrent content branches with progressive multi-token prediction while maintaining competitive parsing accuracy.
- Motivation: Unified VLM parsers improve accuracy and generality but generate full-page outputs through a single sequential trajectory, creating an inference bottleneck for dense documents.This contrasts unified methods with pipeline-based systems that process regions flexibly but use fragmented workflows.
- Related Work: Existing acceleration methods reduce inference cost by compressing visual context, bounding textual KV caches, or increasing decoding parallelism.The introduction identifies context shortening and generation-path shortening as complementary acceleration directions.
- Method: HPD-Parsing exploits the mismatch between globally coordinated page layout and largely localized region content by assigning content decoding to parallel branches.Global layout determines spatial structure, region relationships, and reading order, while each region’s detailed content depends mainly on its corresponding visual evidence.
- Results: HPD-Parsing uses staged adaptation and automated difficulty-aware data curation to maintain competitive parsing accuracy while achieving the best throughput.The introduction attributes these training strategies to accuracy preservation during the transition to hierarchical parallel decoding.
- Method: Hierarchical Parallel Decoding restructures full-page autoregressive generation into globally coordinated localized decoding, with a main layout branch dynamically creating concurrent region-specific content branches.Progressive multi-token prediction further reduces decoding steps within each branch.
2. Related Work
Document parsers are organized into pipeline-based and unified methods, with unified VLM systems improving accuracy but generally retaining token-by-token generation. Recent efficiency efforts reduce per-step computation or shorten sequential decoding, yet long structured outputs still impose substantial latency.
- Pipeline-based methods: Pipeline-based methods decompose parsing into specialized stages, typically combining page-level layout analysis with localized VLM recognition or content decoding.This modular organization enables flexible optimization for different document elements.
- Unified methods: Unified methods jointly formulate layout parsing, reading-order prediction, and content recognition as sequence generation.Recent improvements include large-scale data curation, multi-task supervision, output formatting, and reinforcement learning.
- Decoding bottleneck: Token-by-token content generation causes substantial inference latency for text-dense documents with long structured outputs.This limitation applies whether parsing localized regions or an entire page.
- Efficiency improvements: Recent acceleration methods target either reduced per-step computation or a shorter sequential decoding path.DeepSeek-OCR compresses visual tokens to reduce attention cost, while Unlimited OCR uses Reference Sliding Window Attention to bound the textual KV cache during long-sequence generation.
3. Methodology
HPD-Parsing addresses the sequential decoding bottleneck in unified document parsing by coordinating global layout generation with concurrent localized content branches. It further shortens each branch’s decoding trajectory through Progressive Multi-Token Prediction (P-MTP).
- Motivation: Nearly 500× longer decoding than visual encoding on long-output samples identifies decoder latency, rather than full-page visual processing, as the primary efficiency bottleneck.Encoder latency remains relatively stable as output length increases, while decoder latency grows rapidly and dominates inference cost.
- Hierarchical Parallel Decoding: A main layout branch establishes document structure and reading order, while dynamically assigned content branches decode localized regions concurrently.The layout branch emits routing decisions that fork separate branches for regional content generation.
- Dynamic Branch Forking: Shared-prefix KV-cache reuse avoids repeated visual encoding and common-prefix computation while isolating each region from unrelated textual histories.Each content branch maintains only its incremental content KV cache after inheriting shared visual and structural context.
- Training Objective: Branch-specific supervision trains layout tokens globally but restricts content-branch supervision to localized transcription after <CHILD>.The visual context and structural prefix condition content generation without requiring branches to reproduce shared context.
- Progressive Multi-Token Prediction: 6.6 tokens per step is the average accepted length achieved by P-MTP, which drafts and verifies multiple future tokens in parallel within active branches.P-MTP is integrated into both layout and content branches, shortening intra-branch decoding trajectories alongside inter-branch concurrency.
4. Model Training and Data Curation
HPD-Parsing uses staged training to first acquire broad document-parsing capability, then adapt to hierarchical parallel decoding and improve task-specific output quality. Its training data are built through an automated, difficulty-aware curation pipeline that promotes diversity, reliability, and balanced structural coverage.
- Training strategy: Stage 1 trains on complete full-page parsing sequences to learn text recognition, layout understanding, reading-order modeling, and structured generation.Complete document outputs provide dense page-level supervision before hierarchical parallel decoding is introduced.
- Data curation: The curation pipeline combines feature-based clustering and sampling, multi-model annotation and difficulty estimation, VLM-based refinement, and distribution balancing.It uses PaddleOCR-VL-1.5, MinerU-2.5 Pro, and an intermediate HPD-Parsing checkpoint to construct pseudo-labels and assess difficulty.
- Data curation: Clustering selects representative samples across visual-document clusters, reducing redundancy and preventing repetitive layouts from dominating the curated dataset.This produces a diverse and scalable data pool for subsequent annotation.
- Data curation: Refinement iteratively detects recognition, structural, or formatting errors, regenerates annotations with adjusted prompts, and re-evaluates them until quality criteria or N_refinement attempts are reached.Afterward, samples are balanced across Easy, Medium, and Hard levels and across document elements such as text blocks, tables, formulas, and complex layouts.
- Training data by stage: 2.8 million representative full-page training samples initialize broad parsing capability through feature clustering, filtering, and sampling.Annotations are generated primarily by MinerU-2.5 Pro.
5. Inference Workflow
The inference workflow combines FCFS scheduling with Layout-Coordinated Parallel Decoding and P-MTP to dynamically fork concurrent content branches. Shared-prefix KV-cache reuse accelerates token generation across all branches, as formalized in Algorithm 1.
- Workflow design: The workflow extends FCFS scheduling with Layout-Coordinated Parallel Decoding and P-MTP for hierarchical parallel inference.This design dynamically forks concurrent content branches during decoding.
- Workflow design: Shared-prefix KV-cache reuse enables accelerated token generation across concurrent decoding branches.The workflow shares prefix states while branches proceed in parallel.
- Algorithm 1: Algorithm 1 specifies KV occupancy threshold τ, draft window K, and concurrency cap N_max as inference controls.It outputs decoded sequences for all requests while tracking active processing, waiting, KV-block, queue-status, and parent-child state.
6. Experiments
Experiments show that HPD-Parsing combines competitive parsing accuracy with substantial inference acceleration on OmniDocBench v1.6. Its hierarchical parallel decoding also improves robustness on complex documents and common parsing failures.
- Effectiveness Comparison: With only 1B parameters, HPD-Parsing achieves an overall score of 94.91 on OmniDocBench v1.6, establishing a new state of the art among end-to-end unified parsers.The method combines global layout coordination with localized parallel decoding to preserve page-level structure while reducing difficult-region effects.
- Efficiency Comparison: 4,752.1 TPS and 2.68 PPS increase over the autoregressive baseline by 3.06× and 2.62×, respectively.These gains come from concurrent decoding across layout-coordinated content branches and further decoding-step reduction.
- Efficiency Comparison: As output length grows, HPD-Parsing increasingly outperforms the vanilla autoregressive baseline in decoding steps, request throughput, and single-request latency.The reductions reach up to 18.04× in decoding steps, 3.67× in request throughput under batch size 512, and 5.80× in latency under batch size 1.
- Comprehensive Coverage of Complex Documents: HPD-Parsing accurately parses complex mathematical expressions, including long multi-line formulas, matrix equations, fractions, and densely arranged symbols, while preserving valid LaTeX structures.Representative evaluations also cover table-intensive documents.
- Comparative Advantages over Competing Methods: HPD-Parsing remains effective despite imperfect region localization by using document-level visual and semantic context, mitigating errors from misaligned or incomplete bounding boxes.This contrasts with pipeline-based methods that depend heavily on layout analysis and may exclude relevant content or mix adjacent regions.
7. Conclusion and Future Work
HPD-Parsing is presented as a lightweight, high-throughput document parser based on hierarchical parallel decoding. It combines global layout coordination with concurrent region-level content generation instead of a single autoregressive trajectory.
- Conclusion and Future Work: HPD-Parsing is a lightweight, high-throughput document parser built on hierarchical parallel decoding.The paper presents this paradigm as its central design basis.
- Conclusion and Future Work: The design separates global document coordination from localized content decoding.This division follows the observation that document parsing requires global coordination while content decoding is largely localized.
- Conclusion and Future Work: A main layout branch coordinates the document while concurrent local branches generate region-level content.This hierarchical structure replaces the traditional single autoregressive trajectory.