PaperlayerView exact revision on arXiv

Source-linked AI summary

DeepSeek-OCR: Contexts Optical Compression

Haoran Wei, Yaofeng Sun, Yukun Li

arXiv:2510.18234v1cs.CV

TL;DR

Long textual contexts are computationally challenging, motivating optical compression through visual representations. DeepSeek-OCR combines DeepEncoder with a DeepSeek3B-MoE decoder to compress and reconstruct text, achieving near-lossless OCR around 10× compression and retaining about 60% accuracy at 20×. The authors present this as an early proof-of-concept requiring further validation beyond OCR.

  • Problem

    Long textual contexts face computational challenges from quadratic sequence-length scaling, motivating investigation of visual modality as a more token-efficient textual representation.

  • Method

    DeepSeek-OCR uses DeepEncoder to extract and compress high-resolution visual representations, then a DeepSeek3B-MoE decoder to reconstruct text through OCR-style decoding.

  • Results

    Near-lossless OCR compression is achieved at approximately 10× ratios, while 20× compression retains 60% accuracy; the model also achieves state-of-the-art OmniDocBench performance using the fewest vision tokens.

  • Takeaways & Limitations

    The preliminary findings suggest vision-text compression as a promising direction for long-context processing, historical context compression, and memory-forgetting mechanisms in LLMs.

  • Takeaways & Limitations

    OCR alone is insufficient to fully validate true context optical compression, and the early-stage approach requires further investigation and additional evaluations.

Abstract

from arXiv · show

We present DeepSeek-OCR as an initial investigation into the feasibility of compressing long contexts via optical 2D mapping. DeepSeek-OCR consists of two components: DeepEncoder and DeepSeek3B-MoE-A570M as the decoder. Specifically, DeepEncoder serves as the core engine, designed to maintain low activations under high-resolution input while achieving high compression ratios to ensure an optimal and manageable number of vision tokens. Experiments show that when the number of text tokens is within 10 times that of vision tokens (i.e., a compression ratio < 10x), the model can achieve decoding (OCR) precision of 97%. Even at a compression ratio of 20x, the OCR accuracy still remains at about 60%. This shows considerable promise for research areas such as historical long-context compression and memory forgetting mechanisms in LLMs. Beyond this, DeepSeek-OCR also demonstrates high practical value. On OmniDocBench, it surpasses GOT-OCR2.0 (256 tokens/page) using only 100 vision tokens, and outperforms MinerU2.0 (6000+ tokens per page on average) while utilizing fewer than 800 vision tokens. In production, DeepSeek-OCR can generate training data for LLMs/VLMs at a scale of 200k+ pages per day (a single A100-40G). Codes and model weights are publicly accessible at http://github.com/deepseek-ai/DeepSeek-OCR.

1. Introduction

DeepSeek-OCR investigates visual representations as a compact medium for textual information, using OCR to evaluate vision-text compression. It introduces DeepEncoder and reports strong decoding precision at moderate compression ratios, alongside practical deployment results.

  • Motivation: Quadratic sequence-length scaling makes long textual contexts computationally challenging, motivating visual compression of document text into fewer tokens.A single document image can encode rich textual information with substantially fewer tokens than equivalent digital text.
  • Motivation: OCR provides a quantitative testbed for vision-text compression by mapping visual representations back into textual representations.The task bridges vision and language while enabling measurable compression-decompression evaluation.
  • Results: 96%+ OCR decoding precision is achieved at 9-10× text compression, with approximately 90% at 10-12× and approximately 60% at 20× on Fox benchmarks.The benchmark covers diverse document layouts, and formatting differences may make actual accuracy somewhat higher.
  • Architecture: DeepEncoder maintains low activation memory and few vision tokens for high-resolution inputs by serially connecting window attention, global attention, and a 16× convolutional compressor.The compressor reduces tokens before they enter dense global attention.
  • Practical utility: DeepSeek-OCR achieves state-of-the-art end-to-end performance on OmniDocBench while using the fewest vision tokens and supports charts, chemical formulas, geometric figures, and natural images.The system is also described as capable of generating 33 million pretraining pages per day using 20 nodes with eight A100-40G GPUs each.
  • Implications: The work presents 7-20× token reduction as a preliminary direction for historical context compression and computationally efficient large-scale text processing.The authors frame OCR as a proof-of-concept rather than a complete validation of optical context compression.

2. Related Works

Related work describes three common open-source VLM vision-encoder designs and their deployment or resolution trade-offs, while positioning DeepSeek-OCR within end-to-end OCR models.

  • Typical Vision Encoders: Current open-source VLMs use three main vision-encoder types, each with distinct deficiencies.Figure 2 summarizes these encoder categories.
  • Typical Vision Encoders: Dual-tower encoders such as Vary offer controllable parameters and activation memory but require dual image preprocessing and complicate deployment and training parallelism.The approach uses parallel SAM encoders for high-resolution image processing.
  • End-to-end OCR Models: End-to-end OCR models replace traditional pipelines requiring separate detection and recognition expert models, with Nougat demonstrating dense academic-paper OCR on arXiv.The passage frames end-to-end OCR as a major development in image-to-text systems.

3.1. Architecture

DeepSeek-OCR uses a unified encoder-decoder VLM: DeepEncoder extracts and compresses visual representations, while a sparse MoE decoder generates outputs from image tokens and prompts.

  • Architecture: DeepEncoder is approximately 380M parameters, combining an 80M SAM-base and a 300M CLIP-large connected in series.The decoder uses a 3B MoE architecture with 570M activated parameters.
  • Architecture: The encoder extracts image features, tokenizes and compresses visual representations, while the decoder generates the required result from image tokens and prompts.This forms a unified end-to-end VLM architecture.

3.2. DeepEncoder

DeepEncoder combines perception and visual-knowledge extraction with compression and supports multiple native and dynamic resolutions. These modes vary vision-token counts to test compression ratios and accommodate high-resolution inputs.

  • Architecture of DeepEncoder: DeepEncoder is designed to process high resolutions with low activation, few vision tokens, multiple input resolutions, and a moderate parameter count.The design responds to limitations in current open-source encoders.
  • Architecture of DeepEncoder: DeepEncoder combines window-attention perception with dense-global-attention visual knowledge extraction, using SAM-base and CLIP-large as the main architectures.The components are connected in series, with CLIP receiving output tokens rather than images.
  • Architecture of DeepEncoder: 4096 tokens become 256 after 16× compression, keeping overall activation memory controllable.This is the stated compression example for the encoder pipeline.
  • Multiple resolution support: DeepEncoder supports native and dynamic resolution modes to vary vision-token counts for compression experiments and practical high-resolution processing.Dynamic resolution is enabled through positional-encoding interpolation and is particularly intended for ultra-high-resolution inputs such as newspapers.
  • Multiple resolution support: Native modes pair resolutions and token counts: Tiny 512×512 (64), Small 640×640 (100), Base 1024×1024 (256), and Large 1280×1280 (400).Tiny and Small resize images directly, while Base and Large pad images to preserve aspect ratio.
  • Multiple resolution support: For Base and Large modes, padding preserves the original image aspect ratio, while valid vision tokens are fewer than actual vision tokens.The valid-token calculation uses the original image width and height.
  • Multiple resolution support: Dynamic resolution combines native resolutions; Gundam mode uses n×640×640 local tiles plus a 1024×1024 global view, outputting n×100 + 256 vision tokens.The number of tiles is controlled from 2 to 9, and images smaller than 640 in both dimensions use Base mode.

3.3. The MoE Decoder

The MoE decoder reconstructs text representations from DeepEncoder’s compressed vision tokens through a learned nonlinear mapping. DeepSeek-3B-MoE provides substantial model expressiveness with relatively low inference activation.

  • MoE decoder: The decoder uses DeepSeek-3B-MoE, activating 6 of 64 routed experts and 2 shared experts.Approximately 570M parameters are activated during inference.
  • MoE decoder: DeepSeek-3B-MoE combines the expressive capability of a 3B model with the inference efficiency of a 500M small model.The paper identifies this configuration as suitable for domain-centric OCR VLM research.
  • Decoder function: The decoder reconstructs original text representations from DeepEncoder’s compressed latent vision tokens.The mapping is learned through OCR-style training.

3.4. Data Engine

The data engine prioritizes diverse document OCR while adding structured OCR 2.0 examples and limited general-vision and text-only data. Its annotations cover layouts, text, charts, formulas, geometry, and natural images.

  • OCR 1.0 data: 30M PDF pages covering about 100 languages form the core document dataset.Chinese and English account for approximately 25M pages, while other languages account for 5M.
  • OCR 1.0 data: OCR 1.0 combines coarse annotations for optical-text recognition with fine annotations containing interleaved detection and recognition data.Fine annotations include 2M pages each for Chinese and English, using layout and OCR models.
  • OCR 2.0 data: OCR 2.0 data covers chart, chemical-formula, and plane-geometry parsing tasks.Charts are rendered as image-to-HTML-table tasks, chemical formulas use 5M image-text pairs, and geometry data models line segments.
  • OCR 2.0 data: Chart labels use HTML table format rather than OneChart’s dictionary format, while geometry labels use dictionaries containing line segments and coordinate information.The chart representation saves tokens; each geometry line segment follows the Slow Perception encoding manner.
  • Supporting data: General vision data preserves a general vision interface, but DeepSeek-OCR is not intended to be a general VLM.This data accounts for only 20% of the total training data.
  • Supporting data: Text-only pretraining accounts for 10% of the data, while OCR and general vision data account for 70% and 20%, respectively.All text-only data is processed to 8192 tokens, matching the model’s sequence length.

3.5. Training Pipelines

Training proceeds in two main stages: independently training DeepEncoder, then training the complete DeepSeek-OCR model. The pipeline uses staged component placement and freezing within four-way pipeline parallelism.

  • Training pipeline: The training pipeline first trains DeepEncoder independently and then trains DeepSeek-OCR.Gundam-master is produced by continuing training on a pretrained DeepSeek-OCR model with 6M sampled data.
  • Training DeepEncoder: DeepEncoder training uses OCR 1.0, OCR 2.0, and 100M sampled LAION general data for 2 epochs.Training uses batch size 1280, AdamW, cosine annealing, learning rate 5e-5, and sequence length 4096.
  • Training DeepSeek-OCR: The complete model uses four-way pipeline parallelism, with two parts assigned to DeepEncoder and two to the decoder.SAM and the compressor form a frozen vision tokenizer, while CLIP serves as the input embedding layer.
  • Compression evaluation: Table 2 evaluates vision-text compression on English Fox documents containing 600-1300 text tokens.DeepEncoder outputs 64 or 100 vision tokens after resizing images to 512×512 or 640×640.

4. Evaluation

Evaluation examines compression limits, document parsing efficiency, and broader capabilities across document types and modalities. DeepSeek-OCR maintains strong OCR performance with few vision tokens while supporting multilingual and deep-parsing tasks.

  • Vision-text Compression Study: 100 English Fox documents with 600–1300 tokens were used to test compression–decompression feasibility.Testing used the English document portion and DeepSeek-OCR tokenization.
  • Vision-text Compression Study: 97% decoding precision is achieved within a 10× compression ratio, while precision approaches 60% near 20× compression.Performance declines beyond 10×, potentially because of more complex layouts and blurred long text at limited resolutions.
  • OCR Practical Performance: 100 vision tokens surpass GOT-OCR2.0’s performance using 256 tokens on OmniDocBench.At 400 tokens, DeepSeek-OCR achieves performance on par with state-of-the-art systems.
  • OCR Practical Performance: Slides can achieve satisfactory performance with 64 vision tokens, whereas newspapers require Gundam or Gundam-master mode because their text tokens reach 4–5,000.Book and report documents perform well with 100 vision tokens.
  • Additional Capabilities: Deep parsing handles charts, chemical formulas, simple geometry, and natural images through secondary model calls and a unified prompt.The model also supports nearly 100 languages with layout and non-layout OCR outputs.

5. Discussion

The discussion frames optical compression as a preliminary route to scalable long-context processing and memory-like forgetting. It combines compression ratios with progressive image degradation, while emphasizing that practical implications remain underexplored.

  • Compression Boundaries: 10× compression is described as near-lossless for OCR, while 20× compression retains 60% accuracy.These results define preliminary boundaries for vision-text compression.
  • Memory Forgetting: Progressively resizing rendered images reduces token counts for older contexts, making distant text increasingly blurred while recent information remains higher fidelity.The approach is presented as resembling biological forgetting curves.
  • Scope: The work is early-stage and requires further investigation into the practical implications and limitations of vision-text compression systems.The proposed direction targets scalable ultra-long contexts with differing resolutions across recency levels.

6. Conclusion

The conclusion presents DeepSeek-OCR as a preliminary validation of contexts optical compression and a practical model for large-scale LLM/VLM data production. It also acknowledges that OCR alone does not fully validate true context optical compression.

  • Conclusion: DeepSeek-OCR preliminarily validates contexts optical compression by decoding text exceeding 10 times the quantity of a small number of vision tokens.The report positions this as an initial feasibility result.
  • Conclusion: DeepSeek-OCR is described as a practical model capable of large-scale pretraining data production for LLMs and VLMs.
  • Future Evaluation: OCR alone is insufficient to fully validate true context optical compression, motivating future digital-optical interleaved pretraining and needle-in-a-haystack evaluations.
Loading 2510.18234v1…