Source-linked AI summary
Web Retrieval-Aware Chunking (W-RAC) for Efficient and Cost-Effective Retrieval-Augmented Generation Systems
Uday Allu, Sonu Kedia, Tanmay Odapally, Biddwan Ahmed
TL;DR
RAG systems need chunking that balances retrieval quality, latency, cost, and scalability, while conventional methods struggle with high-volume web ingestion. W-RAC decouples deterministic web parsing from LLM-based semantic grouping over structured identifiers, achieving comparable or better retrieval performance with substantial efficiency gains.
Problem
Traditional chunking methods struggle to preserve semantic integrity while remaining cost-efficient and scalable across heterogeneous, high-volume web content.
Method
W-RAC parses web pages deterministically into structured units with stable identifiers and uses LLMs for retrieval-aware grouping decisions without regenerating source text.
Results
W-RAC achieves comparable recall and ranking quality while reducing output tokens by 84.6%, latency by ∼60%, and total LLM costs by 51.7%.
Takeaways & Limitations
W-RAC provides a more deterministic, debuggable, and extensible chunking paradigm for production RAG systems.
Takeaways & Limitations
Traditional chunking must operate across heterogeneous formats including PDFs, HTML pages, Markdown files, and dynamically generated web content.
Abstract
from arXiv · showhide
Retrieval-Augmented Generation (RAG) systems critically depend on effective document chunking strategies to balance retrieval quality, latency, and operational cost. Traditional chunking approaches, such as fixed-size, rule-based, or fully agentic chunking, often suffer from high token consumption, redundant text generation, limited scalability, and poor debuggability, especially for large-scale web content ingestion. In this paper, we propose Web Retrieval-Aware Chunking (W-RAC), a novel, cost-efficient chunking framework designed specifically for web-based documents. W-RAC decouples text extraction from semantic chunk planning by representing parsed web content as structured, ID-addressable units and leveraging large language models (LLMs) only for retrieval-aware grouping decisions rather than text generation. This significantly reduces token usage, eliminates hallucination risks, and improves system observability.Experimental analysis and architectural comparison demonstrate that W-RAC achieves comparable or better retrieval performance than traditional chunking approaches while reducing chunking-related LLM costs by an order of magnitude.
1 Introduction
Document chunking is foundational to RAG because it shapes how web content is segmented, indexed, and retrieved. W-RAC addresses the cost and scalability limits of conventional approaches by treating chunking as retrieval-aware planning rather than text generation.
- Chunking determines how source content is segmented, indexed, and retrieved in RAG pipelines.
- Fixed-size, rule-based, and agentic strategies face cost, determinism, debuggability, or scalability limitations in high-volume web ingestion.Agentic chunking additionally incurs computational overhead from repeated text generation and transformation.
- W-RAC reframes chunking as a planning problem and uses lightweight semantic planning to produce retrieval-optimized chunks without regenerating source text.
2 Background and Limitations of Traditional Chunking
Traditional chunking methods prioritize simplicity or structural alignment but struggle to preserve semantic integrity while remaining retrieval-effective, cost-efficient, and scalable across heterogeneous web content. These limitations motivate a more efficient retrieval-aware paradigm.
- RAG chunking must balance semantic coherence with retrieval granularity under strict context-length constraints.Chunk quality affects recall, precision, latency, and downstream generation quality.
- Traditional strategies struggle with heterogeneous formats such as PDFs, HTML pages, Markdown files, and dynamically generated web content.
- Fixed-size chunking is simple and inexpensive but can break semantic boundaries, mix unrelated topics, and degrade retrieval relevance.
- Rule-based chunking aligns with document structure but lacks adaptability to varying content density and retrieval requirements.
- Agentic chunking processes raw text with LLMs, creating high token and inference costs, hallucination risks, limited transparency, and poor scalability for continuous web ingestion.
- These limitations motivate a more efficient and retrieval-aware chunking paradigm.
3 Web Retrieval-Aware Chunking (W-RAC)
W-RAC combines deterministic web parsing with structured, ID-addressable representations and LLM-based semantic grouping plans. Original text is resolved locally into final chunks for retrieval processing, supporting transparent and reproducible chunk construction.
- 3.1 Design Principles: W-RAC is guided by preserving source text, retrieval awareness, cost efficiency, determinism, observability, and web-native structure.
- 3.2.1 Deterministic Web Parsing: Web pages are parsed into structured representations, and semantic units such as headings and paragraphs receive stable unique identifiers.
- The LLM receives identifiers, hierarchy, ordering, and optional metadata instead of raw text, then outputs chunk plans as ordered identifier lists.
- The LLM acts as a semantic grouping planner rather than a content generator.
- Chunk plans are resolved locally by mapping identifiers to original text before chunks are assembled, embedded, and indexed.
4 Retrieval Awareness in W-RAC
W-RAC incorporates retrieval considerations into chunk planning by using document structure, length, semantic content, and content type to influence boundaries. The paper states that this design improves recall and precision, with detailed comparisons in Table 1.
- W-RAC explicitly incorporates retrieval considerations into chunk planning.
- Chunk boundaries can reflect heading depth and section hierarchy, token-length constraints, entity density, and semantic cohesion.
- Content type, including tables versus paragraphs, can also influence chunk boundaries.
- The retrieval-aware design aligns chunks more closely with real-world query patterns and improves both recall and precision.Detailed comparisons are presented in Table 1.
5 Evaluation Dataset
RAG-MULTI-CORPUS is a diverse enterprise benchmark spanning multiple document formats, organizations, domains, and query intents. Its design supports evaluating retrieval across factual, reasoning, comparative, and procedural requirements.
- The benchmark contains 236 documents from five fictional organizations and 786 curated query–answer pairs with ground-truth citations.
- Documents cover PDF, Markdown, HTML, DOCX, and PPTX formats, reflecting heterogeneous enterprise knowledge bases.
- Queries are categorized into seven types to cover factual recall, reasoning, comparison, and procedural understanding.
- The query mix enables assessment of chunking effects across intents, especially procedural and comparative questions sensitive to chunk boundaries and semantic coherence.
6 Experimental Results
Across the RAG-Multi-Corpus benchmark, W-RAC reduces token output, processing time, and total cost while maintaining competitive retrieval quality and improving precision across organizations and query types.
- Efficiency and Cost: 59.10%: W-RAC reduces average processing time per file from 9.18 seconds to 3.78 seconds.P90 and P95 latency also decrease by 54.38% and 51.12%, respectively.
- Efficiency and Cost: 49.90%: W-RAC increases average input tokens from 2,447.93 to 3,669.64 while reducing output-token consumption.The additional input encodes structured metadata for semantic planning, and output tokens are priced more highly than input tokens under the stated pricing model.
- Efficiency and Cost: 51.70%: W-RAC lowers complete-pipeline cost for 236 files from $3.64 with Agentic Chunking to $1.75.This corresponds to savings of $1.89 under GPT 4.1 pricing.
- Retrieval Quality: W-RAC consistently improves precision across organizations, including Precision@3 increases from 0.54 to 0.81 for ZX Bank and from 0.46 to 0.76 for Cendara University.The baseline achieves slightly higher recall in some cases, while W-RAC maintains competitive recall and comparable or slightly lower NDCG.
- Retrieval Quality: Across query types, W-RAC improves precision for temporal, comparative, and procedural questions, including Precision@3 increases from 0.43 to 0.79 for temporal queries and from 0.50 to 0.68 for procedural queries.Comparative queries reach Precision@3 of 0.77, and precision improvements are observed across all query types.
7 Conclusion
W-RAC reframes web document chunking as explicit semantic planning over structured, ID-addressable representations rather than text generation. It matches agentic chunking’s retrieval quality while reducing cost and improving operational robustness.
- Conclusion: W-RAC decouples deterministic web parsing from LLM-based grouping decisions over structured, ID-addressable representations.This avoids unnecessary text regeneration, reduces hallucination risk, and improves observability.
- Conclusion: 84.6% fewer chunking-time output tokens, approximately 60% lower end-to-end chunking latency, and 51.7% lower total LLM costs were achieved.The reductions occurred despite a modest increase in input tokens from structured metadata.
- Conclusion: W-RAC achieved comparable recall and ranking quality to agentic chunking while consistently improving retrieval precision across organizations and query types.The evaluation used the RAG-Multi-Corpus benchmark.
- Conclusion: Explicit ID-based chunk plans can be inspected, audited, cached, and recomputed without reprocessing source text.This supports rapid iteration and adaptive retrieval strategies.
- Conclusion: The framework supports extensions including entity-aware chunking, graph-based retrieval, and policy-driven chunk recomposition.These extensions follow from the explicit and extensible chunk-plan design.
- Conclusion: W-RAC offers a production-oriented balance of retrieval quality, cost efficiency, and operational robustness for large-scale heterogeneous web ingestion.The conclusion positions it as an alternative to traditional and agentic chunking approaches.
Chunk Grouping and Hierarchical Structuring Prompt
The chunking instructions combine hierarchical context reconstruction, preservation of related content, and special handling for procedural material. They require output that retains original content or chunk IDs according to the specified prompt variant.
- Chunk Grouping and Hierarchical Structuring Prompt: Chunk Grouping and Hierarchical Structuring Prompt: extract only main policy, explanatory, or instructional content into logical, context-rich units.The prompt targets rules, eligibility, and charges while excluding unrelated page material.
- 1. Three-Level Heading Hierarchy: Three-Level Heading Hierarchy: trace parent_heading relationships to assign Level 1, Level 2, and Level 3 headings, reusing existing IDs when levels are missing.Only existing heading or chunk IDs may be used, and one heading ID may fill multiple levels.
- 2. Parent Headings with Multiple Children: Parent Headings with Multiple Children: include a shared parent heading ID in every child group rather than emitting it as a standalone array.The example repeats heading_66 across both child arrays.
- 3. Procedural Content: Procedural Content: keep step-by-step instructions, sequential how-to guides, and ordered action lists together in one chunk array.The processing steps explicitly require detecting procedures and grouping all steps together.
- 4. Context & Merging: Context & Merging: use headings, titles, parent_heading relationships, and document context to infer hierarchy and merge small contextless fragments.Relevant titles and headings should accompany dependent content, while parent-child relationships may be inferred from sequential patterns.
- 5. Filtering / 6. Output Rules: Filtering and Output Rules: remove cookies, navigation, and logins, then output only chunk IDs in JSON arrays with sufficient contextual headings.Small contextless fragments must not appear as standalone arrays.
- PROCESSING STEPS: Processing Steps: the workflow maps hierarchy, detects procedures, fills three heading levels, repeats shared parents, merges fragments, and groups topical arrays.The required JSON format is {"chunks": [["id1", "id2", "id3"], ...]}.
- Agentic Chunking Prompt / 2. Heading Structure / 3. Content Preservation / 4. Chunking Strategy / 5. Grouping Related Content: Agentic Chunking Prompt: preserve all Markdown content and structure, avoid over-chunking, keep lists and complete elements intact, and format tables as Markdown.Its output uses full two- or three-level heading prefixes and removes empty or no-finding chunks.