Source-linked AI summary
Reading Beyond OCR Noise: Robust Cross-Lingual RAG for Legislative Archives
Radha Krishna Deshpande, Ajoy Mondal, Akash Manna, C V Jawahar
TL;DR
Historical legislative archives pose retrieval challenges because degraded pages and mixed scripts can cause OCR errors that propagate to retrieval and question answering. The paper combines layout-aware OCR with an agentic retrieval loop that checks evidence and rewrites queries when context is insufficient. On the Punjab Vidhan Sabha debates dataset, the authors report resilience to historical vocabulary shifts, degraded text, and severe OCR noise.
Problem
Degraded text, irregular layouts, marginal notes, and mixed scripts can introduce layout-analysis and OCR errors that propagate to retrieval and question answering in historical legislative archives.
Source: §1 · p. 1Method
The framework combines layout-aware OCR with an agentic retrieval loop that checks retrieved evidence, reformulates queries when context is insufficient, and supports grounded answer generation.
Source: §1 · p. 1, §3.3 · p. 4Results
On the Punjab Vidhan Sabha debates dataset, the system demonstrated resilience to historical vocabulary shifts, degraded text, and severe OCR noise.
Source: §6 · p. 13Takeaways & Limitations
The authors conclude that the framework addresses multilingual document-search challenges through iterative query rewriting, adaptive retrieval, and vision-language generation.
Source: §6 · p. 13Severe OCR errors can still affect dense retrieval and entity matching; iterative retrieval and sufficiency checks increase inference latency, and text-to-chunk conversion may lose complex table structure.
Source: §5.3 · p. 13
Abstract
from arXiv · showhide
India's historical public archives present severe computer vision and document image analysis challenges: decades of parliamentary records survive only as degraded, multi-column scanned pages featuring mixed scripts, complex layouts, and dense visual noise where OCR failure is the primary obstacle to information retrieval. We present an end-to-end framework that couples vision-based layout segmentation and multi-script OCR with an agentic retrieval-augmented generation (RAG) pipeline. Our approach specifically addresses how visual degradation and layout fragmentation propagate into text extraction errors, using an iterative agentic loop to navigate around OCR noise and layout ambiguities before generating answers. Evaluated on the 200,000-page Punjab Vidhan Sabha archive, our system demonstrates robust handling of historical document degradation, outperforming standard text-only baselines.
1 Introduction
Historical legislative archives pose retrieval challenges because degraded pages, irregular layouts, marginal notes, and mixed scripts can introduce layout-analysis and OCR errors that propagate downstream. The paper proposes evaluating document-image processing, retrieval, and answer generation together through layout-aware OCR and an agentic retrieval loop.
1 Introduction
- Faded text, irregular layouts, marginal notes, and mixed scripts can cause layout-analysis and OCR errors that propagate to retrieval and question answering.The challenge includes Hindi, Punjabi, and English documents with varying print quality and complex layouts.Source: §1 · p. 1
- The framework combines layout-aware OCR with an agentic loop that checks retrieved evidence, reformulates queries, and searches again when context is insufficient.The authors frame this as an end-to-end pipeline from document images through retrieval to answer generation, rather than treating OCR as separate preprocessing.Source: §1 · p. 1
- The study focuses on Punjab Vidhan Sabha proceedings, a multilingual archive spanning 1950–2024, and uses open-weight models for local deployment and traceable source-page evidence.The introduction motivates evaluating visual processing, retrieval, and answer generation together.Source: §1 · p. 2
- The stated contributions include evaluating cross-lingual question answering on a large historical document collection and analyzing how OCR and layout errors affect retrieval and answer quality.The contribution list also identifies an agentic RAG framework combining layout-aware OCR, hybrid retrieval, and iterative evidence checking.Source: §1 · p. 2
2 Related Work
Related work spans agentic and multilingual RAG, OCR for historical documents, and vision-language document retrieval. These lines of work motivate an approach tailored to noisy, multilingual legislative archives.
2 Related Work
- The related-work framing places multilingual archive navigation at the intersection of RAG, agentic search, multilingual embeddings, and multimodal document retrieval.The authors state that these paradigms were not originally optimized for a corpus like theirs.Source: §2 · p. 2
2.1 Agentic and Multilingual Retrieval-Augmented Generation
- Agentic RAG treats retrieval as an adaptive multi-step loop [20] [24], while multilingual systems face cross-lingual retrieval and output-consistency challenges [5].Prior work also discusses multilingual evidence [14], concurrent multilingual indexing, and multilingual embedding precedents [28].Source: §2.1 · pp. 2–3
2.2 OCR and Historical Multilingual Document Processing
- OCR noise degrades downstream RAG performance, with compounding errors reported for non-Latin and multi-script settings such as Punjabi Gurmukhi and Hindi Devanagari.Historical-text post-correction and domain-specific context for legislative entity recognition are also discussed [3].Source: §2.2 · p. 3
2.3 Multimodal and Vision-Language Document Retrieval
- Vision-language document retrieval encodes page images to preserve layout structures, while sparse lexical search combined with dense visual embeddings is presented as a direction for complex repositories [19] [7] [18] [12].Multi-page retrieval is also covered [6]; the paper describes its pipeline as multilingual textual dense representations paired with an adaptive agentic loop.Source: §2.3 · p. 3
3 Methodology
The methodology processes native-language queries against multilingual document representations, retrieves context, and uses an agent loop to assess sufficiency and rewrite queries when needed. It then generates a grounded response in the user's native language.
3.1 Query Management and Multi-Lingual Processing
- Queries are processed in their native script and language without intermediate machine translation, then the agent loop checks context sufficiency and the generator produces a grounded answer.The described pipeline embeds the query natively and can rewrite it when needed.Source: §3.1 · p. 3, Figure 1 · p. 4
3.2 Offline Processing and Real-Time Retrieval
- Documents are embedded offline into a multilingual dense vector space and stored with metadata; real-time retrieval embeds the native query and returns the top-k context chunks by similarity.The retrieval stage compares the query vector with document vectors.Source: §3.2 · p. 4
3.3 Iterative Agent Loop for Context Evaluation
- An evaluation LLM assesses whether retrieved context is sufficient; insufficient context triggers adaptive query rewriting, and the refined query is sent back through retrieval.The rewriting objective balances semantic similarity to the original query against a conceptual penalty for terms likely to be missed because of OCR errors.Source: §3.3 · pp. 4–5
3.4 Grounded Response Generation
- After context sufficiency is validated, the generation LLM produces a grounded response directly in the user's native language, conditioned on the native query and validated context.The described generation probability conditions each token on preceding generated tokens, the query, and context.Source: §3.4 · p. 5
3.5 Algorithm: Native-Language Agentic RAG Loop
- The algorithm retrieves five context items per loop, evaluates sufficiency, rewrites the query when needed up to a maximum loop count, and generates the native-language response.It stops when context is sufficient or the loop limit is reached.Source: §3.5 · p. 5
4 Experiments and Results
Experiments evaluate the Punjab Vidhan Sabha archive using a stratified benchmark and a smaller hand-transcribed OCR subset. The evaluation covers OCR recognition, retrieval, and answer quality, with BGE-M3 and the optimized pipeline establishing the strongest retrieval performance standard.
4.1 Dataset
- The archive contains approximately 200,000 pages in Hindi, Punjabi, and English, often mixing scripts, with older pages more degraded and recent pages still densely formatted.The corpus spans the 1950s to 2024 and includes proceedings, debates, reports, notices, and other legislative records.Source: §4.1 · p. 6
- Evaluation uses 1,000 stratified pages and 200 gold-annotated question-answer pairs, plus 20 hand-transcribed pages for OCR quality evaluation.The sample is balanced across decade, script, and document type; the 20-page subset spans degradation levels.Source: §4.1 · p. 6
4.2 Experimental Setup
- The full 200,000-page archive is indexed, while quantitative results use the stratified 1,000-page, 200-query benchmark; OCR model combinations are separately evaluated on 20 hand-transcribed pages.Experiments use BAAI/bge-m3 for dense and hybrid token-level multilingual embeddings with support for English, Punjabi Gurmukhi, and Hindi Devanagari.Source: §4.2 · pp. 6–7
4.3 Metrics
- Evaluation covers OCR with CRR and WRR, retrieval with MRR and Hit@k, and answer quality with F1.CRR and WRR are based on Levenshtein edit distance; answer F1 combines precision and recall through their harmonic mean.Source: §4.3 · pp. 7–8
4.4 Retrieval Performance Comparison
- BGE-M3 and the optimized pipeline establish the strongest retrieval performance standard on the PVS benchmark, while larger dense retrievers achieve Hit@3 above 0.81.Token-level and smaller cross-lingual models such as MuRIL and LaBSE yield near-zero or low MRR@1 scores.Source: §4.4 · p. 8
5 Ablation
The ablations compare OCR and layout pipelines and test whether iterative retrieval improves answer quality. They also identify remaining constraints from degraded pages, latency, and tabular structure.
5.1 OCR and Layout Pipeline Evaluation
- 95.67% CRR and 80.03% WRR for fine-tuned YOLO + CRNN compare with 89.00% CRR and 60.35% WRR for CRAFT + CRNN on the 20-page ground-truth subset.Google OCR and Surya OCR score higher, while the proposed pipeline is fully local and offers greater control.Source: Table 2 · p. 11, §5.1 · p. 11
5.2 Ablation on Agent Loop Iterations
- The Fig. 5 example shows a Punjabi query retrieving the correct English document after two irrelevant retrieval iterations and a third query refinement.The evaluation varies the iteration limit from L = 1 to 3 on a fixed 200-query benchmark.Source: §5.2 · pp. 11–12, Figure 5 · p. 12
- Answer F1-Score rises from approximately 0.18 to 0.35 as L increases from 1 to 3, while MRR@1 rises from approximately 0.89 to 0.97 and Precision@3 from approximately 0.33 to 0.37.The study reports the largest effect on answer correctness, nearly a 2× relative gain.Source: §5.2 · p. 12, Figure 6 · p. 13
5.3 Limitations
- Severe OCR errors can still affect dense retrieval and entity matching; iterative sufficiency checks add latency, and text-to-chunk conversion may lose complex table structure.The authors identify OCR robustness, inference latency, and table-structure preservation as future-work targets.Source: §5.3 · p. 13
6 Conclusion
The paper presents Agentic-RAG for multilingual historical archives, combining iterative retrieval with vision-language generation. On the Punjab Vidhan Sabha debates, it reports resilience to degraded text and OCR noise and argues that agentic workflows can improve information discovery in multilingual public records.
6 Conclusion
- Agentic-RAG combines iterative query rewriting, adaptive document retrieval, and vision-language generation for complex historical archives.The conclusion frames it as a bridge between archives and user information needs.Source: §6 · p. 13
- On Punjab Vidhan Sabha debates spanning multiple decades and Punjabi, Hindi, and English, the system showed resilience to historical vocabulary shifts, degraded text, and severe OCR noise.The iterative loop verifies context sufficiency to minimize hallucinations and ground answers in source material.Source: §6 · p. 13
- The authors conclude that agentic workflows can significantly enhance information discovery in multilingual public records.They identify dependable, transparent, and accessible digital archives as a potential broader direction for researchers, policymakers, and the public.Source: §6 · pp. 13–14
- Future work includes multimodal retrieval, expansion to more Indian languages and government archives, conversational search with long-context reasoning, and user studies of usability and reliability.These are proposed future directions, not evaluated outcomes in the conclusion passages.Source: §6 · p. 14