PaperlayerView exact revision on arXiv

Source-linked AI summary

PIXELRAG: Web Screenshots Beat Text for Retrieval-Augmented Generation

arXiv:2606.28344v1

TL;DR

Web RAG depends on brittle, lossy HTML parsing, while screenshot-based retrieval for the open web remains largely unexplored. PIXELRAG retrieves and reads rendered webpages as images at web scale, consistently outperforming text-based RAG across diverse benchmarks, including text-centric tasks. These results support pixel-space RAG as an increasingly practical approach for preserving visual and structural webpage information.

  • Problem

    Web RAG relies on brittle, lossy HTML parsing, while screenshot-based retrieval remains largely unexplored for the open web.

  • Method

    PIXELRAG renders webpages into screenshots, indexes visual embeddings, retrieves image tiles, and feeds them directly to a vision-language reader.

  • Results

    PIXELRAG consistently outperforms no-retrieval and text-based RAG across text-centric, multimodal, noisy-news, and agentic benchmarks.

  • Takeaways & Limitations

    Pixel-space RAG preserves visual and structural webpage cues and becomes increasingly practical as vision-language models improve, with image compression reducing token costs by up to 3×.

  • Takeaways & Limitations

    Screenshot-based retrieval loses actionable hyperlink structure, limiting direct link following for multi-hop reasoning and entity disambiguation.

Abstract

from arXiv · show

1 Introduction

PIXELRAG asks whether web RAG can operate directly on rendered screenshots, bypassing complex and lossy HTML parsing. It introduces a web-scale screenshot-based system and reports broad gains over no-retrieval and text-based baselines, while improving token efficiency through image compression.

  • Motivation: HTML-to-text parsing is complex, engineered, and error-prone [11], while even advanced parsers [14] [15] discard visual cues and structured content such as tables, charts, and images.These losses degrade retrieval indexes and leave readers with noisy, difficult-to-comprehend linearized text.
  • Approach: PIXELRAG performs RAG directly in pixel space by indexing web screenshots, retrieving image tiles, and providing them to a vision-language model without HTML parsing.Its visual pipeline renders webpages, builds a visual index, and retrieves screenshot tiles for the reader.
  • System and scale: PIXELRAG is presented as the first end-to-end screenshot-based RAG system at web scale, covering 7M Wikipedia articles and news from CNN, AP News, and BBC.Its scalable collection pipeline gathers HTML, CSS, and images, reconstructs pages locally, and produces screenshots at scale; retrieval uses Qwen3-VL-Embedding [28] adapted to screenshot data.
  • Results: PIXELRAG consistently outperforms no-retrieval and text-based RAG baselines across text-centric, multimodal, noisy-news, and agentic benchmarks.The gains include Wikipedia QA tasks such as NQ and SimpleQA [31], despite their largely text-answerable questions, as well as LiveVQA [32] and agentic benchmarks [33].
  • Efficiency: Image compression and reader training for lower resolutions improve the accuracy–efficiency Pareto frontier, reducing token cost by up to 3× while maintaining accuracy.Pixel representations create an additional route for improving RAG token efficiency through image compression [34] [35].

2 Related Work & Motivation

Text-based RAG over web data depends on brittle, lossy HTML-to-text parsing that can remove visual cues, structured content, and rendered-only information. Advances in vision-language models motivate pixel-based RAG for large-scale open-domain web retrieval, beyond prior work focused mainly on curated visual documents or limited Wikipedia subsets.

  • Text Parsing from Web Data: HTML-to-text parsing is a critical preprocessing bottleneck because heterogeneous web pages interleave language with tables, figures, charts, infoboxes, and dynamic content [38] [11].Current pipelines typically rely on heuristic extractors, making reliable web parsing difficult at large scale.
  • Text Parsing from Web Data: Parsers are brittle and lossy, discarding visual cues and structured content; one extractor can remove over 40% of recoverable webpage text [38] [11].Parser choice can substantially affect downstream performance in RAG systems [8].
  • Summary: Why Pixel-Based RAG?: Modern VLMs increasingly support pixel-based RAG by understanding text rendered as pixels and becoming more token-efficient, including systems such as DeepSeek-OCR [34] and Glyph [35].VLMs already outperform text-only models on structurally rich content such as tables and infoboxes [41] [42] [43] [16].
  • Visual Document Retrieval: This work extends visual retrieval from small PDF and slide benchmarks [22] [44] [24] [45] [46] [47] and limited Wikipedia efforts [23] [25] to end-to-end RAG over large-scale open-domain Wikipedia.It evaluates whether fully visual RAG can replace text-based RAG on benchmarks including SimpleQA and NQ.
  • Summary: Why Pixel-Based RAG?: Even preserved content loses spatial grouping, font hierarchy, and emphasis when flattened into a one-dimensional sequence, making answer-bearing regions harder to distinguish.Web pages are designed as rendered artifacts, and these visual cues help readers locate information.

3 PIXELRAG

PIXELRAG is an end-to-end web retrieval-and-generation pipeline that operates entirely in pixel space, from screenshot collection and tiling through visual indexing and vision-language generation. It addresses web-scale deployment with offline rendering, fixed-size tiles, and approximate nearest-neighbor retrieval.

  • Pipeline overview: PIXELRAG renders webpages into screenshots, slices them into fixed-size tiles, embeds each tile, indexes the embeddings, and feeds retrieved top-K tiles to a vision-language reader.
  • Data collection: Offline rendering avoids repeated network and browser costs: cached local mirrors make retries free and experiments reproducible, while content-only scrolling captures cover pages independently of length.
  • Data collection: Fixed-width 875-pixel screenshots are split into non-overlapping 1024-pixel tiles, producing ∼30M Wikipedia tiles and ∼3.6M news tiles.
  • Index construction: Because the datastore reaches 30M tiles, PIXELRAG avoids prohibitively expensive multivector retrieval and uses a FAISS IVF index for scalable approximate search and incremental updates.
  • Index construction: ∼2 days: the full offline pipeline processes 7M Wikipedia articles on one machine with 128 CPU cores, 2 TB RAM, and 8 H100 GPUs.

4 Embedding Model Contrastive Learning Pipeline

The pipeline synthesizes out-of-distribution contrastive training triples entirely from the screenshot datastore, using an LLM to generate answerable queries and filter false negatives. It fine-tunes both the language backbone and ViT with InfoNCE, yielding consistent gains on rendered webpage screenshots while completing in under three hours on one H100.

  • Synthetic contrastive data: The training set is synthesized from the datastore without external labels, with an LLM as the sole annotator, and benchmark queries and labels remain unseen during training.Each triple contains a query, a positive page, and hard negatives that share the positive’s structure and topic but do not answer the query.
  • Synthetic contrastive data: Queries are generated from information-dense pages, then filtered for self-containedness and answerability using LLM checks against the source tile.This removes queries that implicitly reference the page and retains pairs only when the LLM can correctly answer from the tile alone.
  • Dynamic hard-negative mining: Top-K neighbors retrieved by the base model become hard-negative candidates, but candidates that also answer the query are identified as false negatives and removed.The filter asks the LLM to classify each candidate as CORRECT, WRONG, or CANNOT_ANSWER, retaining the first M=2 surviving negatives per query.
  • Contrastive training: The model is fine-tuned with InfoNCE loss [49] over mined triples and in-batch negatives, using cosine similarity and a temperature parameter.
  • LoRA fine-tuning with unfrozen ViT: Applying LoRA to both the LLM backbone and ViT brings consistent gains on rendered webpage screenshots, contrary to prior document-image retriever practice [22].The authors hypothesize that visually near-duplicate webpage tiles require stronger visual discrimination; training completes in under 3 hours on a single H100.

5 Evaluation

PIXELRAG outperforms text-based RAG across six benchmarks, with gains in retrieval and end-to-end QA accuracy, especially for structured content. Its advantages extend to modality ablations, agentic search, compression, and stronger VLM readers.

  • 5.2 Main results: PIXELRAG improves end-to-end QA across all six benchmarks, with up to 8.3% higher recall and 4.5% higher accuracy than text-based baselines; fine-tuning adds 5.3% recall and 5.0% accuracy.The largest improvements occur on NQ-Tables, where answers require structured content such as tables and infoboxes.
  • 5.5 Compression: At comparable token budgets, two PIXELRAG tiles match or exceed three text chunks while using fewer tokens, and image resolution provides a controllable compression knob.Downsampling tiles with Lanczos resampling reduces token count proportionally.
  • 5.2 Retrieval analysis: PIXELRAG retrieves answer-bearing evidence more often than Trafilatura, achieving 83.8% versus 77.4% evidence Recall@3, with the advantage concentrated in tables and paragraphs.For tables, evidence Recall@3 is 34.8% for PIXELRAG versus 23.8% for Trafilatura and 5.0% for mwparserfromhell; paragraph retrieval improves by 19.8%.
  • 5.3 Ablations: Screenshot retrieval is consistently better than text retrieval in modality ablations, and it improves downstream accuracy even when readers receive OCR text.Both retrieval and reader input formats matter, but retrieval is the larger factor; Screenshot →OCR outperforms Text →Text on both benchmarks.
  • 5.4 Agentic search: PIXELRAG achieves the highest F1 at the lowest cost for GPT-5 ReAct on MoNaCo, outperforming text retrieval, Google, and DS-Serve while costing 2–4× less.The evaluation uses token-level F1 because MoNaCo answers often contain multivalue lists and partial credit is appropriate.
  • 5.6 VLM scaling: PIXELRAG’s advantage depends on VLM capability: early small VLMs trail text retrieval by over 12.5%, while Qwen3-VL-8B and later models match or beat text by up to 5.9%.The crossover first occurs at Qwen3-VL-4B, where pixel retrieval matches text retrieval.

6 Conclusion

PIXELRAG performs retrieval and generation directly over rendered webpages, avoiding HTML parsing and text extraction. Scaled to over 30M tiles covering all of Wikipedia, it outperforms text-based RAG even on text-centric benchmarks without visual reasoning.

  • Over 30M tiles covering all of Wikipedia enable PIXELRAG to outperform text-based RAG on text-centric benchmarks where visual reasoning is unnecessary.
  • PIXELRAG performs retrieval and generation directly over rendered screenshots, with no HTML parsing or text extraction in the loop.
  • The approach addresses the persistent challenge of extracting clean text from diverse webpages for web-data retrieval-augmented generation.

Technical appendices and supplementary material … A.4 Mini-Datastore for Embedding Ablations

The appendices detail PIXELRAG’s rendering, datastore construction, embedding-training recipe, and efficient ablation evaluation. They specify large-scale corpus coverage, multi-stage data filtering, contrastive-training settings, and a compact mini-datastore for rapid iteration.

  • A.1 Rendering Pipeline: Content-only screenshots are produced by rendering Wikipedia in headless Chromium, removing browser UI and whitespace, then tiling the article area for indexing.Figure 6 illustrates the before-and-after processing.
  • A.1 Rendering Pipeline: Wikipedia storage reaches ∼5.6 TB and news storage ∼469 GB, but images can be discarded after embedding and regenerated for top-K pages at query time.The vector index can additionally be compressed using low-storage methods such as LEANN [66].
  • A.2 Datastore Fetching: The datastore covers 7,134,778 Wikipedia articles with approximately 30M tiles and 667,523 news articles with 3.6M tiles.Wikipedia uses a 2025-08 Kiwix snapshot with 100% content-page coverage; news combines BBC, AP News, and CNN.
  • A.3 Embedding Training: Data Recipe Details and Prompts: Embedding training uses successive appendix-documented recipe stages covering tile selection, synthetic query generation, false-positive filtering, hard-negative judging, and implementation details.The appendix collects the prompts, heuristics, and implementation choices referenced from §4.
  • A.3.1 Knowledge-intensive tile sampling: Sampling removes failed or short renders, metadata and namespace pages, and tiles beyond the first 70% of each page to avoid boilerplate evidence.The render-quality gate uses page_height < 3,000 px, while regex rules exclude categories including Template:, Wikipedia:, File:, Help:, Talk:, Module:, and Draft:.
  • A.3.2 Synthetic query generation prompt: Synthetic queries are generated from rendered tiles by gemini-3.1-flash-lite-preview, which returns either a five-line Q / A / S / T / C block or SKIP.Generation uses temperature=0.7 and max_output_tokens=1024.
  • A.3.3 Self-contained-query filter prompt; A.3.4 Hard-negative consistency-judge prompt; A.3.5 Answerability filter; A.3.6 Training implementation details: The pipeline filters queries with gpt-4o for self-containment, then answerability, and evaluates retrieved hard negatives through answer-and-judge stages using only each candidate tile as visual context.The self-containment filter removes 15.1% of candidates, reducing 195,079 pairs to 165,537; training retains roughly 40K pairs and uses GradCache with a batch size of 64, chunk size 4, two hard negatives, and a 7×10−6 peak learning rate.
  • A.4 Mini-Datastore for Embedding Ablations: A compact mini-datastore samples 400 evaluation queries and combines each gold article’s tiles with the base model’s top-100 retrieved tiles for efficient checkpoint evaluation.This produces a mixed relevant-and-irrelevant candidate pool; the passage states that the resulting datastore contains 400 questions and 7,426 …

A.5 Reader Fine-tuning: Method and Results · B Evaluation Protocol

Reader fine-tuning uses automatically verified retrieval triples to train readers on low-resolution images with distractors, while evaluation compares compression-only and SFT readers across compression factors and retrieved-tile counts. Compression harms the base reader, but SFT substantially recovers performance and can exceed the uncompressed reference.

  • A.5 Reader Fine-tuning: Method and Results: SFT data reuse verified (q, p, a) triples from embedding training without additional annotation, then retrieves k∈{1, . . . , 6} tiles and adds p if absent.Retrieved tiles are downsampled with Lanczos resampling.
  • A.5 Reader Fine-tuning: Method and Results: The training format teaches the reader to process low-resolution images and ignore distractor tiles.This format simulates the retrieval setting by combining retrieved tiles with the relevant passage.
  • A.5 Reader Fine-tuning: Method and Results: The reader is fine-tuned with standard token-level cross-entropy over answer tokens conditioned on the retrieved tile set T and query q.The loss is defined as LSFT = −E(q,T ,a) t log Pϕ(at | T , q, a<t).
  • B Evaluation Protocol: Evaluation uses GPT-4.1 LLM-judge accuracy on a held-out 500-example test set, sweeping compression factor c and retrieved-tile count k.Each compression level compares the no-SFT compression-only reader with the SFT reader, using the uncompressed base reader at 1× as the ceiling reference.
  • A.5 Reader Fine-tuning: Method and Results: 0.947 average accuracy at 2× compression is +9.3pp over compression-only and +4.2pp over the uncompressed 0.905 ceiling.At 3× compression, SFT reaches 0.910, +17.2pp above its compression-only baseline; the supplied passage truncates the remaining comparison.
  • A.5 Reader Fine-tuning: Method and Results: 2× compression lowers the base reader’s average accuracy from 0.905 to 0.854 (−5.1pp), while 3× lowers it to 0.738 (−16.7pp).These results measure the no-SFT reader applied directly to compressed tiles.

B.1 Benchmark Details · B.2 Grading Protocol · C Additional Results

The evaluation spans multiple visual, textual, encyclopedic, and news question-answering benchmarks, with benchmark-specific subsets and exclusions. Wikipedia tasks use deterministic GPT-4.1 LLM-as-judge grading, while LiveVQA uses exact multiple-choice letter matching.

  • B.1 Benchmark Details: Benchmark coverage includes NQ, NQ-Tables, SimpleQA, MMSearch, Encyclopedic VQA, LiveVQA, and MoNaCo, with Wikipedia tasks sharing a 30M-page datastore and LiveVQA using a separate news datastore.Table 7 identifies the datastore scope; MoNaCo uses 1,315 multi-hop Wikipedia questions [56].
  • B.1 Benchmark Details: 749 Google Landmarks v2 [67] Encyclopedic VQA examples use automatic questions, while iNaturalist is excluded because the official release lacks query images.The automatic category is largest and contains naturally phrased questions generated from Wikipedia sections.
  • B.1 Benchmark Details: 300 MMSearch examples are evaluated end to end, comprising 171 image-query cases and 129 text-only cases.This evaluates the complete MMSearch subset rather than a sampled portion.
  • B.1 Benchmark Details: 6,632 LiveVQA QA pairs from CNN, BBC, and AP News use jointly embedded editorial photos and question text, excluding Forbes and Variety because anti-bot protections prevent reliable capture.The full LiveVQA dataset contains 26,888 QA pairs, but the evaluation uses the listed news subset.
  • B.1 Benchmark Details: MoNaCo evaluates all 1,315 multi-hop Wikipedia questions [56] with a GPT-5 ReAct agent and varies only the search backend, keeping the agent loop, prompt, and reader fixed.GPT-5 is used as controller for stronger and more stable agentic behavior, isolating retrieval-backend effects.
  • B.2 Grading Protocol: Wikipedia QA predictions are graded by GPT-4.1 at temperature 0, seed 42, and max_tokens=1000, scoring CORRECT as 1.0 and other labels as 0.0.The grader follows [31] by assigning CORRECT, INCORRECT, or NOT_ATTEMPTED; NQ and NQ-Tables accept matches to any of up to 10 gold aliases.
  • B.2 Grading Protocol: LiveVQA uses exact letter-match grading for its five-option multiple-choice questions rather than LLM-based evaluation.The supplied protocol passage explicitly distinguishes LiveVQA from the GPT-4.1 grading used for Wikipedia benchmarks.

C.1 Results with Alternative Readers … D.4 Detailed Failure Decomposition

Across alternative readers and model scales, PIXELRAG’s advantages remain directionally consistent, including reasoning models, while analyses trace text-retrieval failures to parser information loss, rank displacement, and reader errors. Text extraction is strongest overall with trafilatura, yet its structural limitations motivate pixel-space retrieval for tables and visually organized evidence.

  • C.1 Results with Alternative Readers: 75.1% versus 70.3% at top-3 and 63.7% versus 58.3% at top-1 show LoRA improving SimpleQA accuracy with the Qwen3-VL-4B reader, matching all directional findings across reader variants.Tables 8–9 repeat the main results for the alternative reader.
  • C.2 Full Results of Scaling VLM Performance: +4.8 pp for Qwen3.5-4B reasoning and +4.6 pp for Qwen3.6-35B-A3B reasoning show that the pixel advantage persists across reasoning-mode models, despite higher output-token costs.These trends come from the full 31-model sweep summarized in Table 5 and visualized in Figure 7.
  • D.1 Wikipedia Text Extractor Comparison: trafilatura in Markdown scores 6.40/10 and wins 36 of 50 pages, establishing it as the strongest text parser among seven evaluated Wikipedia extractors.Resiliparse follows at 6.34/10 with 31 wins, and the pipeline uses trafilatura for text-retrieval baselines.
  • D.1 Wikipedia Text Extractor Comparison: Even the best parser reaches only 4.38/10 on tabular data because linearization loses alignment, merged-cell relationships, and spatial grouping that screenshots preserve.This structural ceiling motivates bypassing parsers for table-bearing content.
  • D.2 Visual Information Loss During HTML Parsing: Rendered pages can collapse to near-empty text or flatten merged-cell tables so that visually organized evidence, such as December’s “Manager of the Month,” becomes unrecoverable.Figure 8 contrasts the original rendered pages with trafilatura’s extracted text.
  • D.3 Retrieval Signal Loss Under Text Linearization: In a 12-query MoNaCo trace, text never retrieved an answer-bearing percentage chunk in its top-5, while pixel retrieval placed the rendered table tile at rank 2 with score 0.616.The corresponding same-article text table/reference chunk appeared at rank 38 with score 0.580.
  • D.4 Detailed Failure Decomposition: Reader loss remains after successful retrieval: with Qwen3.5-4B, the reader answered incorrectly on 7.0% of questions where valid PIXELRAG evidence appeared in top-3.This equals 56 of 799 questions and separates reader errors from parser and rank failures.
  • D.4 Detailed Failure Decomposition: Among 91 rank-loss cases, 44% involved text ranking an answerless infobox above answer-bearing content; on affected paragraph cases, PIXELRAG recovered top-3 evidence for 37 of 42 questions (88%).Text’s mean paragraph-evidence rank was 22.5 versus 1.8 for PIXELRAG on the same questions, and the infobox occupied rank 1 for 75.9% versus 67.1% of queries.

D.5 HTML DOM Lookup Baseline: Setup and Analysis

The HTML DOM lookup baseline preserves webpage structure by recovering contiguous raw-HTML spans for retrieved text chunks, but HTML underperforms flat text because markup dilutes reader context. This supports pixel-space retrieval as a more token-efficient way to preserve structure.

  • Setup: The baseline reuses text retrieval, then recovers each chunk’s original HTML from a local Kiwix ZIM archive for reader input.The reader receives concatenated HTML for k=3 retrieved passages, separated by <hr> delimiters.
  • Setup: DOM lookup extracts distinctive cell values or prose fragments, normalizes text, selects the tightest match, and resolves the span to contiguous direct children.This preserves intermediate tables, paragraphs, and lists while removing style, script, and navigation elements; unmatched chunks fall back to flat text.
  • Results: HTML reaches 59.8% vs. 71.6% QA accuracy on SimpleQA and 56.6% vs. 59.0% on LiveVQA, despite retrieval quality differing by less than 1 pp.The gap therefore arises in reading rather than retrieval; oversized passages are truncated to 30k characters to fit the 65k-token reader window.
  • Why HTML hurts: tag dilution: HTML markup expands SimpleQA context from 7,601 to 28,941 characters, causing tag dilution and a larger −11.8 pp accuracy gap than LiveVQA’s −2.4 pp gap.LiveVQA articles are shorter and have simpler DOM structure, reducing markup overhead.
  • Implications: Although HTML is semantically richer than linearized text, screenshots encode equivalent tabular and sectional structure in roughly 875 visual tokens per tile without markup overhead.This token efficiency supports the paper’s central claim that pixel-space retrieval preserves document structure for context-limited readers.

D.6 Directly RAG on Raw HTML Data

The fully HTML-native RAG pipeline preserves DOM structure and achieves comparable retrieval, but raw markup overwhelms the reader, causing substantially lower QA accuracy than plain-text RAG on nearly every benchmark.

  • Setup: The pipeline chunks raw HTML at DOM boundaries, preserves table headers in row-based sub-chunks, filters navigation and reference elements, and indexes 25.7M embeddings.It uses Qwen3-VL-Embedding-2B, FAISS IVFFlat with nlist=4,096 and nprobe=128, and returns raw HTML chunks to the reader.
  • Results: −29.4 pp on NQ and −24.1 pp on NQ-Tables are the largest QA-accuracy gaps for HTML-RAG versus Trafilatura, with drops on every benchmark except EVQA.Retrieval remains comparable or better, so the primary failure is reading raw HTML rather than locating relevant chunks.
  • Results: HTML-RAG improves Recall@3 over Trafilatura by +2.1 pp on NQ and +1.0 pp on NQ-Tables.DOM-boundary chunking yields semantically coherent units, and the embedding model handles HTML markup without difficulty.
  • Implications: The experiment shows that HTML markup is a verbose structural encoding that penalizes context-window-limited readers, whereas screenshots represent structure in approximately 875 visual tokens per tile.Thus, the bottleneck is token efficiency rather than retrieval quality, even with HTML-native chunking and embedding.

E Limitations, Broader Impact, and Future Work · F Prompt Listings

PIXELRAG’s main limitations are lost hyperlink actionability, substantial screenshot storage, English-only coverage, and harder moderation of rendered content, while future work targets broader training and hybrid retrieval. The appendix lists the prompts used for synthetic query generation, filtering, hard-negative construction, and evaluation.

  • E Limitations, Broader Impact, and Future Work: PIXELRAG loses actionable hyperlink structure: links remain visually rendered but cannot be followed to retrieve target pages or support link-based downstream reasoning.The proposed mitigation is to preserve or recover link information alongside pixel representations.
  • E Limitations, Broader Impact, and Future Work: Nearly 6 TB of screenshot tiles occupy the Wikipedia datastore, motivating render-on-demand systems that retain embeddings while re-rendering retrieved content from HTML at inference.This reduces storage after tile embeddings have been computed while preserving access to the relevant source content.
  • E Limitations, Broader Impact, and Future Work: All evaluated datastores are English-only, creating language bias and leaving multilingual corpus extension as an important future direction.The reported sources are English Wikipedia and English-language news outlets.
  • E Limitations, Broader Impact, and Future Work: Screenshot retrieval benefits visually rich webpages by avoiding HTML parsing and text extraction, but it also preserves harmful, misleading, or private rendered content that is harder to filter automatically.Large-scale deployment therefore requires content filtering during rendering or indexing.
  • E Limitations, Broader Impact, and Future Work: Future work includes adapting embeddings to target datastores, training cross-domain models across sources such as scientific papers and forums, and combining text- and pixel-based retrieval scores.These directions are enabled or supported by the synthetic training pipeline and aim to capture complementary signals.
  • F Prompt Listings: The prompt appendix provides the full verbatim prompts used throughout the paper, including synthetic query generation from rendered tiles.Figures 14 and 15 show the tile-to-structured-record generation process and its Stage 1 prompt.
  • F Prompt Listings: The self-contained-query filter labels each question YES or NO and drops questions requiring a specific Wikipedia page, table, or screenshot to understand the request.Figure 16 identifies this as the first false-positive filter in Stage 1.
  • F Prompt Listings: Evaluation prompts construct hard negatives by asking a VLM to answer from candidate tiles, then judging candidates as CORRECT, WRONG, or CANNOT_ANSWER; evidence QA includes text-only and multimodal variants.Candidates judged WRONG or CANNOT_ANSWER are retained as hard negatives, while CORRECT false negatives are dropped.
Loading 2606.28344v1…