PULSAR: Pooled Unified Late-Interaction Search and Retrieval for Enterprise Visual Document RAG

Benjamin Constable; Anup Roy; Vishal Sharma; Rishabh Upadhyay; Robin Mills; Aidan Millar

Source-linked AI summary

PULSAR: Pooled Unified Late-Interaction Search and Retrieval for Enterprise Visual Document RAG

Benjamin Constable, Anup Roy, Vishal Sharma, Rishabh Upadhyay, Robin Mills, Aidan Millar

arXiv:2608.28572v1Aug 28, 2026cs.IR

TL;DR

Enterprise visual-document RAG must preserve chart and layout detail while handling frequently changing, large-scale deal materials, but OCR and figure verbalisation are costly to refresh. PULSAR uses vision-first page-image retrieval with pooled two-stage late interaction, achieving substantially lower search cost while retaining retrieval quality and improving answer-fact recall over the replaced baseline.

Problem
Enterprise decks and board packs contain visual and spatial information that text-first OCR and verbalisation pipelines can lose, while requiring repeated processing as documents change near closing.
Method
PULSAR combines a frozen ColPali-style backbone, pooled two-stage late-interaction retrieval, and event-driven ingestion, decoupling embedding-time from answer-time image resolution.
Results
PULSAR cuts median vector-search latency 15.1× with under 0.01 NDCG@10 and Recall@10 loss, sustains ≈88× an unpooled index’s throughput under load, and more than doubles answer-fact recall over the OCR+verbalisation baseline.
Takeaways & Limitations
The production system makes vision-first late-interaction retrieval affordable at firm scale while preserving retrieval quality and improving factual answer recall.
Takeaways & Limitations
The findings are tied to one firm’s private-markets corpus, baseline, and ColQwen3-4B backbone, and the study does not compare alternative vision-first stacks or hybrids.

Abstract

from arXiv · show

Institutional investors search visually dense pitch decks, board packs, and diligence materials that change hourly near deal closing. OCR followed by figure verbalisation is costly to refresh at this scale and can lose chart detail. We present PULSAR, a production vision-first retrieval system deployed at Mubadala Investment Company. PULSAR indexes page images with a frozen ColPali-style backbone and uses a pooled two-stage late-interaction index: compact page summaries support initial retrieval, followed by exact MaxSim rescoring over a finer pooled representation. On ViDoRe V3, this design reduces median vector-search latency by 15.1 times against an unpooled configuration with less than 0.01 absolute NDCG@10 and Recall@10 loss; production median vector-search latency is 156 ms. Under concurrent load, the pooled index sustains approximately 88 times higher QPS than an unpooled index. The event-driven ingestion path is estimated to be approximately 20 times cheaper per page than the OCR+verbalisation baseline it replaced. Since March 2026, PULSAR has served 78 thousand documents and approximately 2.4 million pages across more than 3,000 deals. At the production top K, it more than doubles answer-fact recall over the OCR+verbalisation baseline.

1 Introduction

Enterprise visual document RAG must preserve tables, chart annotations, legends, and spatial cues while keeping refresh costs manageable at firm scale. PULSAR addresses this with vision-first page indexing and a pooled late-interaction retrieval system.

Motivation: Corporate decks and board packs contain visual and spatial information that text-first pipelines can lose, even with image verbalisation.The standard alternative combines OCR or layout extraction, text chunking, optional VLM captions, and hybrid retrieval.
Motivation: Three processing passes make OCR+verbalisation roughly an order of magnitude more work per page than one vision-first embedding pass at firm scale.The comparison includes document OCR+layout analysis, per-figure VLM verbalisation, and embedding.
Vision-first retrieval: Vision-first indexing makes the page the unit of indexing and grounding, with cost scaling by page count rather than visual complexity.Ingestion uses one forward pass of the retriever’s embedding model per page and omits separate OCR+layout and captioning passes.
Late interaction: Late interaction keeps many patch vectors per page, preserving addressable chart values and layout cues but creating billions of indexed vectors at firm scale.Per-query scoring cost grows linearly with patches per page.
PULSAR: PULSAR combines a frozen ColPali-style backbone, pooled two-stage retrieval, and event-driven ingestion to scale vision-first retrieval in production.Its contributions include separate retrieval and answer-time resolutions and pooled retrieval components.
PULSAR: 15.1× lower median vector-search latency is achieved with under 0.01 NDCG@10 and Recall@10 loss, while ingestion is ≈20× cheaper per page than OCR+verbalisation.The system was deployed at Mubadala across 78k documents, more than 3,000 deals, and ∼2.4M pages, with higher end-to-end factuality metrics than the baseline.

2 Related Work

Related work established late interaction for fine-grained token matching and extended it to page images, while multivector indexing introduced a scaling challenge. PULSAR builds on these lines rather than introducing a new retrieval primitive.

Vision-first document retrieval: ColBERT represents each document with one vector per token and scores query-document pairs by summing each query token’s best document-token match.ColPali extends this approach to page images by treating image patches as tokens.
Vector indexing: PULSAR uses a pretrained ColQwen3-4B without firm-specific fine-tuning and reports beating the OCR+verbalisation baseline on the firm’s decks.Its ANN backend is HNSW with quantisation, while late interaction makes each page multivector rather than single-vector.

3 PULSAR System Design

PULSAR’s system design separates efficient page ingestion and retrieval from higher-resolution answer grounding. It uses hashing, pooled multivector search, and exact MaxSim reranking to control late-interaction cost.

System overview: PULSAR uses a single backbone pass for page embeddings, separates retrieval DPI from answer-time DPI, and scores pooled representations rather than full patch grids.The design keeps exact MaxSim affordable while retaining higher-resolution images for answering.
Embedding backbone: ColQwen3-4B is the deployed ColPali-style multivector retriever selected as the strongest Apache-2.0 model within the half-A10 serving budget.A higher-scoring 8B sibling exceeded that budget.
Ingestion path: Document- and page-level hashes skip unchanged work, changed pages are re-embedded and upserted, and the document hash is stamped after every page becomes durable.This ingestion path also removes orphan pages when a document shrinks.
Ingestion path: Priority queues let analyst uploads preempt background synchronization, with evicted jobs resuming from the last durable page.A custom dispatcher and vLLM divide scheduling across document and GPU-request layers.
Resolution decoupling: 150 DPI renders feed embeddings while separate 500 DPI renders are stored for answer-time VLM grounding.Bounded producer–consumer rendering keeps resident memory bounded by the active window rather than document length.
Pooling options: Mean row/column pooling creates compact page summaries, while hierarchical pooling retains more detail for reranking at higher vector count.Pool factor 3 retains approximately one-third of vectors, with 97.8% NDCG@5 retention reported by prior work.
Pooled two-stage retrieval: Stage 1 uses 8–16 binary-quantised mean-pooled vectors and two HNSW prefetches to form a shortlist; Stage 2 applies exact MaxSim to hierarchical-pooled vectors.Exact MaxSim is reserved for the shortlist rather than all indexed vectors.
Pooled two-stage retrieval: 14× lower index RAM footprint results from discarding full per-page patch vectors after ingestion while retaining pooled vectors.The top-k pages are returned after reranking.

4 Experimental Setup

The experimental baseline is an OCR+verbalisation retrieval stack that converts page content into indexed text for dense and keyword hybrid search.

Baseline: The OCR+verbalisation baseline combines per-page OCR+layout extraction, query-time VLM figure verbalisation, semantic chunking, dense embeddings, and HNSW hybrid retrieval.Only filter-kept verbalisations are indexed before dense and keyword retrieval.

5 Results

PULSAR selects 150 DPI for retrieval and combines pooled late interaction with binary quantisation to reduce search cost while preserving retrieval quality. The resulting index substantially improves latency, concurrency, freshness, and answer-fact recall over the OCR+verbalisation baseline.

5.1 Embedding-DPI ablation: 150 DPI is selected because every higher DPI from 200–600 is equivalent to 150 on ViDoRe V3, while 72 and 100 fall outside the ±0.01 NDCG@10 band.The DPI choice minimizes ingestion cost among quality-equivalent settings.
5.2 Pooling and quantisation ablation: 15.1× speedup reduces median vector search latency to 49.9 ms from 752.7 ms, with quality losses of 0.0088 NDCG@10 and 0.0074 Recall@10.M+Hqp is the lowest-latency configuration, and all nine nonbaseline configurations pass the quality gate.
5.3 Storage and latency costs: 14× lower RAM footprint makes the pooled M+Hqp index deployable at production scale, where an unpooled index would exceed cluster RAM.The pooled index retains only quantised mean-pooled prefetch vectors in RAM.
5.3 Storage and latency costs: 10.9× lower single-query median latency lets pooled search reach approximately 47 QPS under load, while unpooled search remains near 0.5 QPS and drops over half its requests at 64-way concurrency.In production, the pooled index records 156 ms median vector-search latency over a rolling seven-day window, peaking at 62 QPS.
5.5 Ingestion latency and refresh lag: 2.0 s median change-to-searchable lag shows skip-aware ingestion refreshes only changed pages rather than re-embedding whole documents.The p95 lag is 43 s, while the busiest hour reached 63 s p95 and 328 s worst case.
5.6–5.7 Answer quality: 500 DPI eliminates page-legibility errors, and PULSAR beats OCR+verbalisation on Completeness and context- and answer-fact recall at every tested k.At k=1, AnswerFactRecall is approximately 5× the baseline (.49 vs .10); at production k=5, PULSAR exceeds the baseline’s k=10 on every metric.

6 Conclusion

PULSAR combines vision-first retrieval with pooled late interaction to reduce serving cost while preserving retrieval quality. Its evaluation reports improved answer-fact recall over the OCR+verbalisation baseline.

PULSAR uses a pooled two-stage late-interaction index with 15.1× lower median vector-search latency and under 0.01 NDCG@10 and Recall@10 loss.The design also sustains approximately 88× an unpooled index’s throughput under load.
PULSAR’s ingestion path is estimated to be approximately 20× cheaper per page than the OCR+verbalisation baseline.
PULSAR more than doubles answer-fact recall over the OCR+verbalisation baseline.Table 2 reports results on SME-authored investment questions with gold answers and repeated evaluation passes.

Limitations

The reported findings are bounded by the corpus, baseline, backbone, and evaluation design. The paper explicitly cautions that late interaction remains heavier than single-vector retrieval and that its comparisons do not establish universal superiority.

The findings are tied to one chart- and table-heavy private-markets corpus, one OCR+verbalisation baseline, and the ColQwen3-4B backbone.The paper states that performance and cost gaps may shift with corpus complexity, backbone choice, or a stronger text-extraction stack.
Late interaction remains heavier than single-vector retrieval despite pooling and two-stage scoring.
The answer-quality comparison uses 75 SME-authored questions and does not compare alternative vision-first stacks or generate-and-encode hybrids.The comparison is human-validated, but the authors do not claim late-interaction page-image retrieval is best for the task.

Ethical Considerations

PULSAR is presented as an advisory production system with priority-aware infrastructure and explicit operational safeguards. Its deployment configuration spans benchmark, development, and production settings while keeping foreground search ahead of background embedding.

Ethical safeguards: PULSAR is surfaced to analysts as an advisory aid, and outputs entering client-facing artefacts undergo human review.The paper does not recommend deploying vision-first RAG as an unsupervised investment decision-maker.
Priority-aware ingestion scheduling: Page hashes let evicted ingestion jobs resume from the last durable page, bounding wasted work by the in-flight batch rather than document length.
Deployment configuration: Table 4 describes Kubernetes sizing with fractional-A10 inference pods, CPU work on Standard_D8s_v3, and a 512 GiB Qdrant volume for the benchmark.
Priority-aware ingestion scheduling: Search queries enter at top priority, so foreground search is never blocked by background embedding.The scheduler also ages low-priority work to prevent starvation.

E Retrieval Throughput Under Load

Under concurrent load, the pooled index maintains throughput and request completion while the unpooled index saturates. Single-query measurements likewise show a substantial pooled latency advantage, with production median vector-search latency reported separately.

Concurrent throughput: Figure 5 compares throughput and request completion across a 4–64-way concurrency sweep for both indexes.
Single-query latency: 175 ms pooled median latency versus 1.9 s unpooled establishes a 10.9× single-query gap.The cited single-query measurements isolate latency from queueing on the 106,074-page development slice.
Single-query latency: The pooled index’s latency advantage widens through the tail, with p95 intervals of 405–621 ms pooled versus 4,351–8,748 ms unpooled.

G Answer-time DPI Legibility Errors

The section reports page-legibility errors used to assess answer-time DPI, alongside an ingestion-latency figure whose x-axis is document length.

Table 8 reports per-deck page-legibility error counts for two firm investment-committee decks as answer-time DPI varies.
Figure 6 plots end-to-end ingestion latency against document length on a dev cluster.It shows rolling median, IQR, p10–p90, and p99 latency statistics.
Figure 6 clips the document-length x-axis at the 97th percentile, corresponding to 143 pages.

H Evaluation Question Taxonomy

The evaluation taxonomy labels each question along financial-subject and question-operation axes, while a blind SME subsample checks agreement with the LLM judge.

Evaluation Question Taxonomy: The n=75 evaluation questions receive one label on each of two axes: financial subject and question operation.
Financial Subject: Financial-subject categories include operating and financial metrics, market and competitive context, valuation and returns, capital structure and funding, and segment and revenue mix.
Question Operation: Question-operation categories include value extraction and cross-entity or benchmark comparison, with value extraction covering 54 questions (72.0%).
Human Validation: Three blind SME raters re-scored a deal-stratified 20% subsample, and their ratings showed strong agreement with the LLM judge.The table reports Pearson r and quadratic-weighted κ for Completeness, plus Pearson r for AnswerFactRecall.