Source-linked AI summary
Token-Efficient Data Reasoning Agents via Adaptive Structuring of Unstructured Data
Milad Rezaei Hajidehi, Qitong Wang, Stratos Idreos
TL;DR
Complex questions over unstructured documents require costly, repeated evidence gathering, while fully structuring everything in advance is infeasible. The paper introduces agentic data cracking, which extracts reusable structure from documents already opened during reasoning. On FanOutQA, it cuts cost by 53% with one related-question reuse opportunity while preserving accuracy.
Problem
Agents can answer complex questions over unstructured data, but repeated document opens are expensive and advance extraction is infeasible because useful structure is workload-dependent.
Method
Agentic data cracking forks a sub-agent from already-loaded document context to extract, validate, and catalog grounded structure for reuse by future queries.
Results
53% lower cost is achieved on FanOutQA with one related-question reuse opportunity while preserving accuracy.
Takeaways & Limitations
The accumulated structured store can become a durable cross-query substrate for reasoning over repeatedly accessed unstructured documents.
Takeaways & Limitations
The current system assumes a static corpus, with handling of document edits left to dropping or incrementally refining affected cracked objects.
Abstract
from arXiv · showhide
Valuable data remains embedded in unstructured sources: web pages, reports, contracts, filings, earnings calls, and PDFs. The big bet in enterprise AI is deploying LLM agents that reason over this data to answer complex questions for every knowledge worker. Agents can do this today, but at prohibitive cost. Each question repeatedly opens large documents to recover scattered evidence, consuming up to a million tokens. However, if the data were already structured, the same question would reduce to a cheap database lookup. For example, on FanOutQA benchmark, reasoning over an ideal pre-structured store is 28X cheaper, and the gap grows to orders of magnitude as questions fan out over more documents. Yet structuring everything in advance is not viable: documents hold vastly more possible structure than any workload will use, and the useful structure and documents are unknown until queries arrive. We propose agentic data cracking, a method that structures unstructured data adaptively and speculatively as a byproduct of reasoning itself. Structuring is adaptive because observed queries decide when it happens and what matters, and speculative because it goes beyond the current question. Whenever the agent opens a document to answer, a cracking sub-agent forks from the already-loaded context at marginal cost and extracts grounded structure likely to serve related future queries. Over time, an increasing share of queries is fully covered by structured data and answered without opening a document, keeping agentic accuracy at close to RAG cost. On FanOutQA, extended with merely one related question per test question, cracking cuts cost by 53% while preserving accuracy. Agentic data cracking is a first step toward next-generation data infrastructure for agentic reasoning over unstructured data: a shared substrate beneath the model where knowledge that reasoning already paid to uncover accumulates.
1 Introduction
Complex questions over unstructured data require agents to assemble latent structure from scattered evidence, but repeated document reading is expensive. Agentic data cracking adaptively extracts reusable structure during reasoning, reducing later costs while preserving accuracy.
- Motivation: Data reasoning assembles entities, attributes, and relations from sparse evidence across many documents rather than retrieving a single passage.The resulting answer is a latent structured dataset constructed step by step.
- Motivation: Agentic systems achieve higher answer quality than parametric knowledge or fixed top-k retrieval, but repeatedly opening documents makes reasoning costly.The workload is largely prefill-intensive.
- Opportunity: 28× cheaper reasoning is possible when FanOutQA evidence is manually placed in an ideal database instead of raw documents.The benchmark questions span seven documents on average, and savings can reach orders of magnitude with larger fan-out.
- Opportunity: Semantic locality means related queries reopen overlapping documents and reveal which structures matter, while cached prefixes reduce the marginal cost of additional generation.This motivates speculative extraction beyond exact-answer caching.
- Approach: Agentic data cracking forks a sub-agent from already-loaded document context to extract grounded entities, attributes, and relations for future structured reads.The system validates and catalogs these objects, reuses them on later queries, and falls back to documents on misses.
- Results: 53% lower cost is achieved on FanOutQA with one related-question reuse opportunity while preserving accuracy.The ideal store is 28× cheaper, and a case study reports a 3× reduction.
2 Agentic Reasoning: Capable but Expensive
The paper studies latent structured data reasoning, where agents construct intermediate knowledge from unstructured documents to answer multi-step questions. Agentic systems support this process through planning and tool use, but conventional retrieval and parametric knowledge are less accurate.
- Task Definition: Latent structured data reasoning extracts knowledge from unstructured documents, constructs intermediate data and evidence, and passes those artifacts across reasoning steps.The answer comes from reasoning over constructed intermediate data rather than retrieving one fact.
- Running Examples: The paper’s running questions include NBA award winners’ career rebounds and cast members who worked with both Alfred Hitchcock and Martin Scorsese.The examples illustrate multi-step reasoning over many Wikipedia pages.
- Agentic Solutions: Agentic systems plan sequences of steps and adaptively invoke search, document access, code execution, and intermediate-state storage.These capabilities imitate the data reasoning process over large document collections.
- Agentic Solutions: Conventional retrieval and parametric knowledge achieve low accuracy on data reasoning tasks, whereas agentic systems choose tools based on gathered evidence.The comparison motivates the paper’s focus on making agentic reasoning more efficient.
3 Agentic Data Cracking: Adaptive Structuring of Unstructured Data
Agentic data cracking uses query-driven, speculative extraction to build reusable structure from documents during reasoning, reducing the cost bottleneck without opening documents solely for extraction.
- Data systems techniques address data reasoning’s cost bottleneck by substantially reducing token consumption.
- The system forks a cracking branch after a document enters context, extracting grounded entities, attributes, and relations likely to support future queries.The branch reuses the document’s KV-cache prefix and runs outside the answer path.
- Observed queries drive adaptive, incremental organization for future reuse, while the method creates useful structure where none previously existed.This distinguishes agentic data cracking from database cracking, which reorganizes existing relations.
- Cracked objects are evidence-backed RDF-style edges representing entity-to-entity or entity-to-value relations with cardinality, units, and source evidence.The schema supports singular or list-valued relations and records the source document and supporting region.
- Structured reads resolve subjects and relations through a catalogue, returning compact values when available and falling back to raw-document access otherwise.This lets the reasoning agent reuse cracked objects without opening or prefilling raw documents.
4 Evaluation
The evaluation tests whether cracking reuses structure across related questions while preserving answer quality, using FanOutQA and an evolving Hitchcock investigation. Cracking substantially lowers token use and cost, with gains varying according to reuse.
- The evaluation measures inference-cost reduction and answer-quality preservation when related questions have overlapping but distinct objectives and entity sets.
- The experiments use Claude-Haiku-4.5 on FanOutQA and a Hitchcock case study, measuring prefill tokens, decode tokens, API cost, and accuracy.Prompt caching is enabled, and the baseline differs only by the cracking interface.
- Each FanOutQA test question receives one human-verified related question targeting a distinct attribute over overlapping entities, creating a reuse opportunity.Primary metrics remain on the original questions, while the related questions populate the cracked-object store.
- Cracking cuts mean prefill from 189K to 87K tokens on FanOutQA and from 565K to 161K on the case study, while decode stays below 3K tokens.
- Mean cost falls from $0.26 to $0.12 on FanOutQA and from $0.81 to $0.27 in the case study.The case study gains more because twenty related questions precede the ten evaluated questions.
- Median per-question cost is $0.246 under the baseline versus $0.072 with cracking, a 3.4× ratio of medians.
- Cracking is 9× cheaper at the 10th percentile but costs 1.24× the baseline at the 90th percentile, reflecting questions without reuse.It pays off on three quarters of questions and wins clearly in aggregate.
5 Related Work
Agentic data cracking connects deep-research agents with data-management techniques by accumulating reusable, evidence-backed structure over unstructured corpora. It occupies a middle ground between upfront construction and per-query extraction, while persisting across models and queries.
- Agentic data cracking complements deep-research agents and data-management operators by accumulating reusable, evidence-backed structure for later queries.
- Unlike upfront knowledge-base construction, cracking creates structure adaptively as queries reveal what matters; unlike query-driven analytics, it speculates beyond the current request.
- Cracked structures are plain text, allowing them to survive model upgrades, transfer across LLMs, and compound into an organizational asset.
- The approach can be viewed as agent memory with speculation and prefetching, storing corpus structure rather than user preferences or past answers.Extraction cost is amortized across future queries.
6 Discussion
The system constrains read functions and assumes a static corpus, while current benchmarks use open Wikipedia content rather than private organizational documents or long-running query histories.
- Design Decisions, Limitations, and Future Work: The system exposes constrained read functions instead of full SQL, leaving counting, aggregation, joins, and direct multi-hop queries for future work.This design keeps the interface small, fallbacks explicit, and generated queries reliable.
- Design Decisions, Limitations, and Future Work: The current system assumes a static corpus, with edits proposed for handling by dropping affected cracked objects or incrementally refining them.
- Design Decisions, Limitations, and Future Work: The benchmarks use open, commonly trained-on Wikipedia content, not private office documents that are unseen by the model and repeatedly queried across workflows.
- Design Decisions, Limitations, and Future Work: Future benchmarks should capture long histories of queries over documents, ideally from deployed systems, to reflect real-world workloads.Such benchmarks could expose opportunities to improve individual queries and long-running systems.
7 Conclusion
The paper presents agentic data cracking as a way to reduce the cost of reasoning over unstructured corpora by reusing structure extracted during document access. It positions the approach as an initial step toward adaptive data systems for AI reasoning, with applicability wherever systems repeatedly reason over large documents.
- 7 Conclusion: Agentic data cracking reduces token usage and API cost while preserving answer quality by reusing structure extracted from documents already opened during reasoning.
- 7 Conclusion: Related queries reuse extracted structure, while cracking reuses the document’s KV cache, exploiting locality at both levels.
- 7 Conclusion: The remaining gap to an ideal fully structured store suggests an opportunity for data systems to improve agentic reasoning as useful structure emerges across queries.
- 7 Conclusion: The idea applies wherever systems repeatedly reason over large unstructured documents.