Source-linked AI summary
Unlimited OCR Works
Youyang Yin, Huanhuan Liu, YY, Qunyi Xie, Chaorun Liu, Shiqi Yang, Shaohua Wang, Zhanlong Liu, Hao Zou, Jinyue Chen, Shu Wei, Jingjing Wu, Mingxin Huang, Zhen Wu, Guibin Wang, Tengyu Du, Lei Jia
TL;DR
Long-horizon OCR is limited by decoder KV caches that grow with output length, unlike human copying with fading working memory. Unlimited OCR replaces DeepSeek OCR’s decoder attention with R-SWA and combines it with a compressed encoder. The resulting system parses dozens of document pages in one forward pass under a 32K maximum length while maintaining constant KV-cache behavior.
Problem
Long-horizon OCR is constrained because decoding contexts make KV caches grow continuously, increasing memory use and limiting generation speed and length.
Method
Unlimited OCR replaces all DeepSeek OCR decoder attention with Reference Sliding Window Attention while retaining its high-compression DeepEncoder.
Results
Unlimited OCR parses dozens of paper pages in one forward pass and achieves 93% on OmniDocBench v1.5, 6% above the DeepSeek OCR baseline.
Takeaways & Limitations
R-SWA keeps the KV cache constant while reducing attention computation and memory footprint, and is proposed for parsing tasks beyond OCR.
Takeaways & Limitations
Under a finite context length such as 32K, Unlimited OCR remains constrained by the prefill length as pages accumulate.
Abstract
from arXiv · showhide
Recently, end-to-end OCR models, exemplified by DeepSeek OCR, have once again thrust OCR into the spotlight. A widely held view is that employing a large language model (LLM) as the decoder allows the model to leverage the prior distribution of language, leading to improved OCR performance. However, the downside is equally evident: as the output sequence lengthens, the accumulated KV cache drives up memory consumption and progressively slows down generation. This stands in stark contrast to humans, who exhibit no such decline in efficiency during long-horizon copying tasks. In this technical report, we propose Unlimited OCR, a model designed to emulate human parsing working memory. Taking DeepSeek OCR as the baseline, we replace all attention layers in the decoder with our proposed Reference Sliding Window Attention (R-SWA), which reduces attention computation costs while maintaining a constant KV cache throughout the entire decoding process. By combining the high compression rate of DeepSeek OCR's encoder with our constant KV cache design, Unlimited OCR can transcribe dozens of pages of documents in a single forward pass under a standard maximum length of 32K. More importantly, R-SWA is a general-purpose parsing attention mechanism - beyond OCR, it is equally applicable to tasks such as ASR, translation, etc. Codes and model weights are publicly available at http://github.com/baidu/Unlimited-OCR.
1. Introduction
Unlimited OCR targets long-horizon parsing by replacing full-history or reset-at-each-page processing with a human-inspired attention pattern. Its R-SWA mechanism keeps relevant reference information while limiting output context, enabling efficient multi-page OCR.
- Motivation: Current OCR models process fewer than ten pages in one forward pass and often reset memory page by page.The paper contrasts this with humans’ continuous cognitive state, where distant outputs fade while nearby context tracks progress.
- Reference Sliding Window Attention: R-SWA attends to all visual and prompt reference tokens while restricting output attention to the preceding n tokens, with n defaulting to 128.This lets each token perceive the full image while tracking OCR progress through causal state transitions.
- Reference Sliding Window Attention: R-SWA maintains a constant KV cache during inference, reducing attention computation and memory pressure for long-horizon decoding.The mechanism is designed to avoid progressively increasing cost as generated output lengthens.
- Results: 93% on OmniDocBench v1.5 exceeds the DeepSeek OCR baseline by 6%.The result is reported after replacing the decoder attention mechanism with R-SWA while retaining DeepSeek OCR’s high-compression encoder.
- Scope: Unlimited OCR parses dozens of paper pages in a single forward pass, while R-SWA also applies beyond OCR to parsing tasks such as ASR and translation.The paper presents this as a preliminary validation of linear-complexity attention for long-horizon parsing.
2. Related Works
Related work has moved OCR toward unified end-to-end models, but long-document inference remains constrained by visual prefixes and growing decoder KV caches. Unlimited OCR addresses this efficiency setting with a compressed encoder and bounded-cache attention.
- OCR architectures: Traditional document OCR commonly uses detection, recognition operators, and heuristic steps such as cropping and rectification.End-to-end OCR instead merges detection and recognition into one unified function that can parse a page in a single forward pass.
- OCR architectures: End-to-end OCR increases demands on model capacity and training because text detection and recognition are unified.This motivates continued architectural and training research for dense document parsing.
- High-compression encoder: DeepEncoder achieves a 16× image-token compression rate, making it suitable for multi-page long-horizon OCR.Insufficient compression lengthens visual prefixes and can hinder decoding speed and effective decoding length.
- High-efficiency decoder: Decoder inference cost is shaped by LLM activation and KV-cache size, while current models’ caches grow continuously with decoding context.DeepSeek OCR uses a MoE design with 3B total and 500M activated parameters, but its growing cache still limits speed and length.
- High-efficiency decoder: Unlimited OCR uses an encoder plus an MoE-LLM decoder whose attention mechanisms are all R-SWA, with a queue-based KV cache.The queue evicts the KV pair at position m+1 whenever a new token is generated, preventing computational and memory costs from progressively increasing.
3. Methodology
Unlimited OCR replaces DeepSeek OCR’s decoder attention with R-SWA, preserving global prefix access while restricting generated-token attention to a causal sliding window. This design targets long-horizon parsing by bounding attention and KV-cache growth.
- Encoder and decoding bottleneck: DeepEncoder compresses a 1024×1024 PDF image to 256 visual tokens, reducing the prefix burden for long-document decoding.The encoder uses 16× token compression and retains Base and Gundam resolution modes.
- Encoder and decoding bottleneck: A 1:10 visual-to-text compression ratio means 10K visual tokens can require more than 100K output tokens for roughly 20–30 pages.This creates substantial KV-cache and attention-computation demands for vanilla LLM-driven OCR.
- Attention computation: R-SWA uses a fixed prefix window of length L_m and a causal decode window of width n, so its accessible attention set remains bounded.The prefix remains globally visible, while the decode region slides over previously generated tokens.
- KV cache management: For long sequences, R-SWA bounds KV-cache requirements at L_m + n instead of allowing standard MHA cache growth to increase linearly with T.The decode-side cache retains the full prefix and only the most recent n generated tokens.
- Kernel study: Figure 3 reports growing Flash Attention v3 latency for DeepSeek OCR but constant duration for Unlimited OCR as decoding proceeds.The latency stability is attributed to replacing standard attention with R-SWA across the decoder layers.
4. Experimental Settings
Unlimited OCR is trained on approximately 2 million document OCR samples packed to 32K tokens. Training continues from DeepSeek OCR while freezing the DeepEncoder and updating the LLM parameters.
- Data construction: Approximately 2 million OCR samples are constructed with a 9:1 ratio of single-page to multi-page data.Single-page annotations come from Paddle OCR, while multi-page examples are synthesized by concatenating single-page data.
- Data construction: All training data are packed into sequences of 32K tokens.Multi-page data use a separator between concatenated pages.
- Optimization: Training continues from the DeepSeek OCR checkpoint for 4,000 steps with global batch size 256 on 8×16 A800 GPUs.The maximum sequence length is 32K, with random packing used for all data.
- Optimization: The DeepEncoder is frozen during training, and only the LLM parameters are updated.The reported rationale is that DeepEncoder was already sufficiently optimized in DeepSeek OCR.
5. Evaluation
Unlimited OCR is evaluated on standard document-parsing benchmarks, detailed document subcategories, and multi-page one-shot OCR. It achieves strong single-page accuracy, higher throughput, and sustained long-horizon performance.
- Benchmark and Metrics: OmniDocBench evaluates text, formulas, tables, and reading order using task-specific metrics plus a weighted overall score.The in-house benchmark additionally reports Distinct-n and Edit Distance.
- Main Results: 93.92% overall metric on OmniDocBench v1.6 establishes Unlimited OCR as end-to-end SOTA.On v1.5, text edit distance drops by 0.035 and table TEDS improves by 5.96% versus DeepSeek OCR.
- Main Results: 5580 TPS versus DeepSeek OCR’s 4951 TPS under Base DeepEncoder mode represents a 12.7% speed increase.The benchmark’s average documents are relatively short, so the longer-output advantage is expected to be more pronounced.
- Subcategory Study: Across nine OmniDocBench v1.5 document types, Unlimited OCR shows consistent gains across every metric versus DeepSeek OCR.It also surpasses DeepSeek OCR 2 on seven-ninths of both text edit-distance and reading-order scores, without disadvantage on complex layouts.
- Long-horizon Parsing: At 40+ pages, multi-page one-shot OCR maintains edit distance below 0.11 and 97% Distinct-35.The evaluation covers books of 2, 5, 10, 20, and 40+ pages, with at least ten books per category.
- Long-horizon Parsing: Unlimited OCR’s fixed KV cache enables continuous parsing across tens to hundreds of pages while keeping output latency constant.This addresses output-length limits and latency growth that hinder previous long-horizon OCR systems.
6. Efficiency Analysis
Efficiency analysis compares output throughput across increasing sequence lengths. Unlimited OCR maintains the long-output speed advantage that DeepSeek OCR loses as generation grows.
- Throughput Scaling: At 6,000 output tokens, DeepSeek OCR trails Unlimited OCR by 35% in TPS.At 256 tokens, the two models are virtually equal; DeepSeek OCR’s TPS then steadily declines as output length increases.
- Throughput Scaling: Unlimited OCR’s R-SWA-based inference speed remains more consistent as output length grows.The comparison fixes prefill length at 10 and holds other settings identical under ideal concurrency conditions.
- Throughput Scaling: Consistent generation speed is identified as a critical requirement for long-horizon OCR tasks.The result links throughput scaling directly to the practical demands of extended document parsing.
7. Limitation and Future Work
Unlimited OCR is not truly unlimited under a finite context length because the prefill length constrains how many pages can be processed. Future work targets longer contexts, prefill retrieval, and transfer to ASR and translation.
- Limitation: Finite context length, including 32K, limits Unlimited OCR through the growing prefill length as pages accumulate.The limitation concerns the input-side prefill rather than the fixed KV cache during decoding.
- Future Work: Near-term work will train models with 128K context lengths to support prefilling more pages.This is presented as a short-term response to the prefill constraint.
- Future Work: Long-term work proposes a prefill pool that automatically fetches KV chunks to simulate page-by-page human reading.The stated goal is truly unlimited OCR rather than merely longer fixed-context parsing.
- Future Work: The authors also plan to transfer R-SWA to reference-based tasks such as ASR and translation.This extends the proposed attention mechanism beyond OCR within the stated future-work scope.
8. Conclusion
The report presents R-SWA and Unlimited OCR for long-horizon parsing, reporting lossless parsing-task performance after replacing decoder attention with causal reference-based SWA.
- R-SWA replaces standard decoder attention with causal reference-based sliding-window attention for long-horizon parsing.The report presents this replacement as the basis of Unlimited OCR.
- The model maintains useful information by continuously passing historical outputs into a limited window, implementing a soft form of forgetting.This behavior is described as consistent with human transcription.
- Parsing-task performance remains lossless after replacing all standard decoder attention with causal reference-based SWA.
- R-SWA is intended to reduce attention computation and memory footprint and may extend beyond OCR to other long-horizon parsing tasks.The report identifies attention computation and memory footprint as bottlenecks in long-horizon parsing.
9. Author List
The author list identifies core contributors and additional contributors, with project-leader and technical-director roles marked separately.
- An asterisk denotes the project leader, while a dagger denotes the technical director.
- Youyang Yin, Huanhuan Liu, and YY are listed as core contributors.
- Qunyi Xie, Chaorun Liu, Shiqi Yang, Shaohua Wang, Zhanlong Liu, Hao Zou, Jinyue Chen, Shu Wei, Jingjing Wu, Mingxin Huang, Zhen Wu, Guibin Wang, Tengyu Du, and Lei Jia are listed as contributors.